-- Logs begin at Wed 2022-08-31 18:26:37 UTC, end at Wed 2022-08-31 19:49:57 UTC. -- Aug 31 18:58:08.017240 dv-cinder-835384-26-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 31 18:58:08 2022] *** Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: nodename: dv-cinder-835384-26-iscsi Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: machine: x86_64 Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: clock source: unix Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: pcre jit disabled Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: detected number of CPU cores: 4 Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: current working directory: / Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: detected binary path: /usr/bin/uwsgi-core Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: your processes number limit is 47680 Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: your memory page size is 4096 bytes Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: detected max file descriptor number: 2048 Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: lock engine: pthread robust mutexes Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: thunder lock: enabled Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 31 18:58:08.034483 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 31 18:58:08.071569 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: Python main interpreter initialized at 0x5654975f99c0 Aug 31 18:58:08.071569 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: python threads support enabled Aug 31 18:58:08.071569 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: your server socket listen backlog is limited to 100 connections Aug 31 18:58:08.071569 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: your mercy for graceful operations on workers is 90 seconds Aug 31 18:58:08.072453 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: mapped 403077 bytes (393 KB) for 2 cores Aug 31 18:58:08.072571 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: *** Operational MODE: preforking *** Aug 31 18:58:08.072840 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: *** uWSGI is running in multiple interpreter mode *** Aug 31 18:58:08.072840 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: spawned uWSGI master process (pid: 81612) Aug 31 18:58:08.073032 dv-cinder-835384-26-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 31 18:58:08.073632 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: spawned uWSGI worker 1 (pid: 81626, cores: 1) Aug 31 18:58:08.073967 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: spawned uWSGI worker 2 (pid: 81627, cores: 1) Aug 31 18:58:08.074123 dv-cinder-835384-26-iscsi devstack@keystone.service[81612]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 31 18:58:12.388101 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 18:58:12.388101 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: from cryptography.utils import int_from_bytes Aug 31 18:58:12.388888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 18:58:12.388888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: from cryptography.utils import int_from_bytes Aug 31 18:58:12.389028 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 18:58:12.389028 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: from cryptography.utils import int_from_bytes Aug 31 18:58:12.389487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 31 18:58:12.389487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: from cryptography.utils import int_from_bytes Aug 31 18:58:12.438994 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 18:58:12.439177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 18:58:12.439365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] command line args: [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 18:58:12.439521 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 31 18:58:12.439637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 18:58:12.439720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 31 18:58:12.439802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ================================================================================ {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 18:58:12.439897 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] command line args: [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 31 18:58:12.440057 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] admin_token = **** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.440164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 31 18:58:12.440304 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] config_dir = [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.440393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ================================================================================ {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 31 18:58:12.440493 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.440575 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] admin_token = **** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.440676 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] config_source = [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.440810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] config_dir = [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.440891 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] debug = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.440998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.441081 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.441364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] config_source = [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.441364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] debug = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.441557 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.441661 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.441780 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.441882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.441973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.442070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.442169 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.442267 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.442358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] list_limit = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.442447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.442536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] log_config_append = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.442637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.442739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] list_limit = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.442836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] log_dir = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.442927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] log_config_append = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.443023 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] log_file = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.443114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.443114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] log_dir = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.443292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.443375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] log_file = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.443489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.443568 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.443680 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.443758 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.443878 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.443953 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.444064 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.444197 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.444281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.444362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.444431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.444587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.444669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.444782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.444902 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.445004 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.445116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.445256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.445377 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.445489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.445614 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.445720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.445864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.445945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.446111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.446186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.446386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.446473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.446668 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.446877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.447626 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] publish_errors = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.447723 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.65/identity {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.447859 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.447942 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] publish_errors = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.448090 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.448221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.448300 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.448419 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.448498 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.448614 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.448705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.448782 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.448927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.449008 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] standard_threads = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.449159 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] standard_threads = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.449269 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.449416 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.449492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.449612 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.449689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.449759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] use_journal = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.449892 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.449970 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] use_json = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.450045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] use_journal = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.450114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] use_stderr = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.450185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] use_json = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.450305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] use_syslog = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.450386 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] use_stderr = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.450462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] use_syslog = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.450530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.450673 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 31 18:58:12.450748 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.450925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.451009 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.451159 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.451240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.451413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.451488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.451649 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.451734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.451892 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.451972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.452126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.452206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.452365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.452447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.452597 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.452670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.452841 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.452922 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.453084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.453168 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.453343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.453418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.453572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.453663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.453817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.453901 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.454051 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.454134 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.454294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.454372 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.454533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.454626 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.454765 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.454841 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.455014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.455096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.455263 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.455415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.455487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.455553 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.455724 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.455799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.455971 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.456047 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.456228 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.456302 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.456476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.456549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.456713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.456790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.456941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.457016 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.457168 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.457263 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.457422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.457499 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.457653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.457728 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.457897 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.457978 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.458129 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.458204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.458356 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.458438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.458584 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.458665 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.458826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.458910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.459069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.459142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.459300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.459373 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.459539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.459615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.459765 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.459838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.459997 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.460072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.460227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.460302 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.460453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.460527 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.460680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.460754 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.460899 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.460971 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.461120 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.461214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.461375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.461458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.461617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.461693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.461843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.461916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.462075 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.462149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.462300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.462382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.462552 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.462653 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.462788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.462887 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.463020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.463120 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.463248 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.463348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.463483 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.463566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.463716 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.463795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.463949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.464066 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.464179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.464263 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.464408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.464482 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.464634 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.464707 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.464879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.464962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.465111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.465212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.465366 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.465453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.465595 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.465679 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.465852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.465940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.466084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.466158 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.466312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.466394 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.466547 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.466634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.466780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.466856 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.467041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.467117 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.467277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.467422 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.467524 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.467630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.467759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.467883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.467992 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.468075 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.468226 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.468302 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.468455 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.468537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.468700 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.468788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.468933 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.469010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.469168 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.469264 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.469416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.469514 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.469653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.469734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.469890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.469965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.470123 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.470210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.470355 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.470433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.470587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.470663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.470821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.470908 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.471052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.471129 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.471307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.471383 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.471545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.471628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.471784 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.471865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.472017 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.472098 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.472253 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.472407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.472481 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.472549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.472712 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.472788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.472942 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.473026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.473171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.473283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.473421 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.473503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.473650 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.473725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.473875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.473951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.474104 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.474178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.474333 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.474407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.474567 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.474643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.474797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.474871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.475040 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.475114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.475271 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.475354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.475544 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.475625 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.475786 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.475889 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.476019 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] auth.external = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.476115 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.476250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.476351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] auth.external = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.476548 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.476626 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.476783 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.476859 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.477014 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] auth.password = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.477092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.477268 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] auth.token = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.477348 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] auth.password = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.477511 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.477588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] auth.token = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.477751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.477830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.477990 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.478065 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.478255 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.478330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.478486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.478564 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.478733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.478811 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.478963 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.479039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.479198 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.479274 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] credential.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.479433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] credential.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.479509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.479678 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.479754 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.479927 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.480003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.480171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.480247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.480412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.480489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.480649 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.480732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.480881 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.480957 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.481126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.481283 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.481403 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.481500 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.481629 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.481737 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.481881 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.481984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.482151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.482253 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.482396 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.482497 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.482648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.482750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.482904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.483009 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.483195 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] federation.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.483329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.483438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.483533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] federation.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.483634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.483729 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.483851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.483979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.484076 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.484168 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.484345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.484474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.484599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.484712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.484813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.484920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.485015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.485107 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.485294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.485428 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.485540 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.485660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.485826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.485933 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.486041 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.486138 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.486230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.486399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.486503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.486597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.486690 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.486790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.486889 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.487023 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.487133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.487273 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.487381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.487543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.487660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.487768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.487925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.488039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.488135 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.488251 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.488395 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.488506 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.488629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.488732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.488856 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.488968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.489096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.489288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.489452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.489577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.489684 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.489780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.489873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.490049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.490213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.490362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.490482 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.490592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.490702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.490824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.490961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.491135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.491298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.491456 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.491576 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.491708 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.491825 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.491941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.492042 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.492171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.492271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.492373 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.492470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.492569 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.492669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.492763 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.492763 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.492956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.493051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.493143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.493263 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.493390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.493555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.493689 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.494398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.494497 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.494571 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.494630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.494795 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.494872 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.495039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.495194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.495264 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.495323 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.495497 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.495573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.495736 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.496055 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.496245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.496337 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.496475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.496586 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.496715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.496818 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.496953 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.497043 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.497212 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.497297 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.497455 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.497544 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.497684 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.497750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.497916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.497985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.498165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.498232 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.498408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.498522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.498640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.498710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.498880 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.498965 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.499108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.499176 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.499341 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.499408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.499586 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.499664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.499813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.499901 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.500060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.500170 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.500673 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.500783 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.500879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.500973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.501081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.501178 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.501303 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.501397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.501536 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.501536 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.501766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.501865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.501960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.502053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.502165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.502308 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.502418 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.502514 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.502609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.502703 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.502796 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.502891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.503014 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.503123 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.503258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.503373 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.503511 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.503636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.503759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.503873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.503987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.504100 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.504212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.504307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.504408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.504504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.504596 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.504689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.504810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.504914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.505277 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.505437 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.505437 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.505722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.505874 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.506025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.506189 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.506341 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.506444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.506574 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.506716 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.506867 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.507031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.507176 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.507278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.507416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.507568 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.507675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.507836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.507950 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.508079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.508079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] resource.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.508301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.508399 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.508502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.508604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.508699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.508798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.508893 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.508987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.509088 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] resource.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.509200 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.509303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.509556 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.510186 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.510313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.510440 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.510556 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.510680 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.510804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.510934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.511064 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] role.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.511354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.511487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] role.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.511487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.511700 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.511813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.511944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.512079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.512220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.512350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.512474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] role.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.512572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.512675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] role.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.512771 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.512865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.512957 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.513057 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.513153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.513323 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.513424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.513542 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.513654 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.513766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.513877 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.514015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.514197 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.514381 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.514516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.514629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.514741 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.514862 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.514862 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.515072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.515183 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.515293 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.515433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.515632 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.515819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.515996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.516124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.516249 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.516249 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.516494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.516616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.516715 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.516809 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.516902 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.517004 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.517099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.517209 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.517209 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.517469 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.517605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.517731 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.517854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.517987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.517987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.518205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.518205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.518411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.518522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.518642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.518773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.518873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.518966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.518966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.519152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.519152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.519326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] token.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.519419 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.519512 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.519653 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.519814 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.519814 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.520085 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.520205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.520317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.520437 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.520551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.520663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.520790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.520905 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.521015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] token.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.521124 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.521257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.521361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.521513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.521648 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.521648 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.521897 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.522026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.522157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.522283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.522402 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.522520 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.522649 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.522779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.522888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.522972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.523094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.523168 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.523239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.523317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.connection = **** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.523390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.523459 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.523578 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.523652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.523722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.523800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.523872 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.523942 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.524053 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.524129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.524200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.524337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.524425 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.524574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.524690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.524816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.524957 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.connection = **** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.525093 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.525251 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.525346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.525504 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.525583 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.525746 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.525825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.525988 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.526137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.526219 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.526291 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.526450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.526528 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.526686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.526772 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.526923 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.527002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.527155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.527232 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.527326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81627) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 18:58:12.527413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.527622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.527858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.528105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.528343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.528588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.528820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.529068 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.529335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 31 18:58:12.529507 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81626) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 31 18:58:12.557245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.557475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.557703 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.558274 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.558509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.558734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.566411 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.567165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.594869 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.595101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.595660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.595897 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.649946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.650445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.699772 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.699953 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.700092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.700330 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.854116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.854390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.854621 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.854845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.855066 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.855285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.858442 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.858681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.858908 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.859132 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.859352 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.859575 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.879663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.879805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.879890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.879985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.880116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.880200 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.880345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.880428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.880567 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.880651 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.880780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.880862 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.880993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.881084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.892189 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.892300 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.905055 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.905206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.905289 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.905382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.905522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.905607 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.905737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.905816 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.905946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.906026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.906140 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.906224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.906352 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81627) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.906443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81626) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 31 18:58:12.923692 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.923904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 18:58:12.924134 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.924252 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.924335 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 18:58:12.924455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 18:58:12.924675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.924757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.924877 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 31 18:58:12.924984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.925275 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.925425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.925540 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.925693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.925956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.926218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.935715 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.935999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.936168 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.936272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.936448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.936541 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.936721 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.937004 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.943576 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:12.944296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:12.946514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:12.946806 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.947119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.947258 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:12.947409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.947547 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.947695 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.947833 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.947962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.948120 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.948235 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.948403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.948516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.948664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.948791 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.948932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.949066 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.949345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.949429 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.949622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.949938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.950170 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.975440 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.975616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.976911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:12.977085 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:12.977257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.977398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.982393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:12.982488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:12.982682 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.982764 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.982966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.983050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.989319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:12.989550 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:12.989635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.989839 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.993669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:12.993904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:12.994657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.994780 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.994946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.995054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.995232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.995317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.995514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.995596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.995815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.996338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.996622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.997802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.998099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:12.998390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.013874 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.013971 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.014148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.014233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.018003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.018112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.021075 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.021329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.021416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.021625 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.025731 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.025956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.028824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.030320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.031930 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.032234 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.032526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.032816 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.033101 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.033753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.035099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.035408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.035706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.035990 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.036278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.036576 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.050054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.050350 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.050623 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.050891 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.054830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.055259 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.055651 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.056050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.058287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.059957 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.061531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.061813 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.064456 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.064747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.065044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.065353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.065654 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.068185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.070700 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.073250 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.073652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.077051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.077371 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.077670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.077955 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.078044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.078240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.078411 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.078522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.078821 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.078976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.079393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.079822 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.094845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.096767 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.097245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.097762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.097912 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.098048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.098327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.098614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.098899 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.100809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.102270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.104318 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.104603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.104905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.111515 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.113007 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.115909 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.116213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.116502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.116783 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.121913 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.124873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.125077 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.125395 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.125682 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.129448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.131720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.132635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81626) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.132924 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.133231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.133516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.133793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.134067 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.134301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.134384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.134808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.134898 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.134973 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.135047 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.135179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.135455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.135719 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.135986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.136261 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81626) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.141317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.142801 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.145805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.146098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.146393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.146714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.155070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.155362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.155643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.162545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81627) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 31 18:58:13.162832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.163112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.163386 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.163657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.163929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.164219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.164488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.164773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.165041 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.165338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.165602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.165967 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.166235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81627) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 31 18:58:13.170923 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5654975f99c0 pid: 81626 (default app) Aug 31 18:58:13.174386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996107bd-5651-4309-b832-adef5b2aab51 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:13.174684 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996107bd-5651-4309-b832-adef5b2aab51 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:13.174959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996107bd-5651-4309-b832-adef5b2aab51 None None] PATH_INFO: `/v3` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:13.179113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1/1] 10.222.0.65 () {50 vars in 791 bytes} [Wed Aug 31 18:58:13 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:13.201636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5654975f99c0 pid: 81627 (default app) Aug 31 18:58:19.831556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e0f5c-ed54-4419-9912-7647aed337da None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:19.832060 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e0f5c-ed54-4419-9912-7647aed337da None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:19.832164 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48e0f5c-ed54-4419-9912-7647aed337da None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:19.833478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 2/2] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 18:58:19.841359 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615b40c-49a3-418a-9449-c85fdaef580c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:19.841762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615b40c-49a3-418a-9449-c85fdaef580c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:19.841893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615b40c-49a3-418a-9449-c85fdaef580c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:19.911693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: INFO dbcounter [None req-6615b40c-49a3-418a-9449-c85fdaef580c None None] Registered counter for database keystone Aug 31 18:58:19.931042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG oslo_db.sqlalchemy.engines [None req-6615b40c-49a3-418a-9449-c85fdaef580c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81627) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 18:58:19.937287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writer thread running {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 18:58:20.113543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG passlib.handlers.bcrypt [None req-6615b40c-49a3-418a-9449-c85fdaef580c None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81627) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 18:58:20.113861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG passlib.handlers.bcrypt [None req-6615b40c-49a3-418a-9449-c85fdaef580c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81627) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 18:58:20.172101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6615b40c-49a3-418a-9449-c85fdaef580c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:20.173737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6615b40c-49a3-418a-9449-c85fdaef580c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:20.300395 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1/3] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:19 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 31 18:58:20.731512 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e0f5c-ed54-4419-9912-7647aed337da None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:20.800348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: INFO dbcounter [None req-f48e0f5c-ed54-4419-9912-7647aed337da None None] Registered counter for database keystone Aug 31 18:58:20.821345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG oslo_db.sqlalchemy.engines [None req-f48e0f5c-ed54-4419-9912-7647aed337da None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81626) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 31 18:58:20.828100 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writer thread running {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 31 18:58:20.945683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48e0f5c-ed54-4419-9912-7647aed337da None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:20.950570 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a19f5c-f41e-438c-b0e5-beb81d9389b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:20.951191 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a19f5c-f41e-438c-b0e5-beb81d9389b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:20.951465 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a19f5c-f41e-438c-b0e5-beb81d9389b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:20.951731 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a19f5c-f41e-438c-b0e5-beb81d9389b6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:20.959719 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a19f5c-f41e-438c-b0e5-beb81d9389b6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:21.012817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: WARNING keystone.common.rbac_enforcer.enforcer [None req-90a19f5c-f41e-438c-b0e5-beb81d9389b6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 18:58:21.013157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a19f5c-f41e-438c-b0e5-beb81d9389b6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:21.022507 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 3/4] 10.222.0.65 () {58 vars in 1324 bytes} [Wed Aug 31 18:58:20 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 18:58:21.028570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615b40c-49a3-418a-9449-c85fdaef580c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:21.043586 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615b40c-49a3-418a-9449-c85fdaef580c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:21.047983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc1eb6-10b7-49bf-aa05-9e936c69e5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:21.049083 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1eb6-10b7-49bf-aa05-9e936c69e5b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:21.049408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1eb6-10b7-49bf-aa05-9e936c69e5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:21.049679 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1eb6-10b7-49bf-aa05-9e936c69e5b3 admin admin] PATH_INFO: `/v3/projects/8d8519390e3f4536964a76bc41b6e43c` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:21.051117 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc1eb6-10b7-49bf-aa05-9e936c69e5b3 admin admin] RBAC: Authorizing `identity:get_project(project_id=8d8519390e3f4536964a76bc41b6e43c)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:21.104775 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d5bc1eb6-10b7-49bf-aa05-9e936c69e5b3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 31 18:58:21.105156 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc1eb6-10b7-49bf-aa05-9e936c69e5b3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:21.109685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 2/5] 10.222.0.65 () {56 vars in 1207 bytes} [Wed Aug 31 18:58:21 2022] GET /identity/v3/projects/8d8519390e3f4536964a76bc41b6e43c => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:22.993106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a497e-0188-42c3-83ae-4c80b6310e63 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:22.993775 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a497e-0188-42c3-83ae-4c80b6310e63 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:22.993924 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a497e-0188-42c3-83ae-4c80b6310e63 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:22.998795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 4/6] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 18:58:23.004110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccb4a8-6723-47d0-a9f1-923f2331f945 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:23.004540 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccb4a8-6723-47d0-a9f1-923f2331f945 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:23.004660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccb4a8-6723-47d0-a9f1-923f2331f945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:23.024950 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-32ccb4a8-6723-47d0-a9f1-923f2331f945 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:23.026533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-32ccb4a8-6723-47d0-a9f1-923f2331f945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:23.034454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 3/7] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:23 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 31 18:58:23.300381 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a497e-0188-42c3-83ae-4c80b6310e63 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:23.313403 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a497e-0188-42c3-83ae-4c80b6310e63 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:23.317636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d6606c-1aa0-4790-a2fd-f2d8aaa0c0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:23.318231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6606c-1aa0-4790-a2fd-f2d8aaa0c0f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:23.318509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6606c-1aa0-4790-a2fd-f2d8aaa0c0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:23.318780 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d6606c-1aa0-4790-a2fd-f2d8aaa0c0f0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:23.326269 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6606c-1aa0-4790-a2fd-f2d8aaa0c0f0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:23.328028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d6606c-1aa0-4790-a2fd-f2d8aaa0c0f0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:23.336631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 5/8] 10.222.0.65 () {58 vars in 1324 bytes} [Wed Aug 31 18:58:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 18:58:23.341939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccb4a8-6723-47d0-a9f1-923f2331f945 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:23.350998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccb4a8-6723-47d0-a9f1-923f2331f945 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:23.355871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35919383-990f-49fd-a938-a9d927f155e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:23.356487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35919383-990f-49fd-a938-a9d927f155e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:23.356766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35919383-990f-49fd-a938-a9d927f155e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:23.357037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35919383-990f-49fd-a938-a9d927f155e9 admin admin] PATH_INFO: `/v3/users/2d817dbb81e84447ac0c66c6621a6e8f` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:23.358374 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35919383-990f-49fd-a938-a9d927f155e9 admin admin] RBAC: Authorizing `identity:get_user(user_id=2d817dbb81e84447ac0c66c6621a6e8f)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:23.360181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35919383-990f-49fd-a938-a9d927f155e9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:23.361790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 4/9] 10.222.0.65 () {56 vars in 1198 bytes} [Wed Aug 31 18:58:23 2022] GET /identity/v3/users/2d817dbb81e84447ac0c66c6621a6e8f => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:25.322867 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35923e4-85ce-48c6-a118-52aacce98995 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:25.323378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35923e4-85ce-48c6-a118-52aacce98995 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:25.323498 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35923e4-85ce-48c6-a118-52aacce98995 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:25.324718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 6/10] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:25.329858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec236a96-8a52-477d-8a56-5bcb30f98fd9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:25.330263 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec236a96-8a52-477d-8a56-5bcb30f98fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:25.330489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec236a96-8a52-477d-8a56-5bcb30f98fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:25.348683 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ec236a96-8a52-477d-8a56-5bcb30f98fd9 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:25.349895 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ec236a96-8a52-477d-8a56-5bcb30f98fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:25.367998 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a6aed-8964-4a86-8463-c2f315cc9d06 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:25.368279 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a6aed-8964-4a86-8463-c2f315cc9d06 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:25.368554 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4a6aed-8964-4a86-8463-c2f315cc9d06 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:25.369929 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 7/11] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 18:58:25.375264 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c4381-05cd-4daf-b009-db840bedb933 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:25.375540 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c4381-05cd-4daf-b009-db840bedb933 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:25.375807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c4381-05cd-4daf-b009-db840bedb933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:25.417329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG passlib.handlers.bcrypt [None req-7b4c4381-05cd-4daf-b009-db840bedb933 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81626) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 31 18:58:25.417942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG passlib.handlers.bcrypt [None req-7b4c4381-05cd-4daf-b009-db840bedb933 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81626) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 31 18:58:25.431274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 5/12] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:25.467220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7b4c4381-05cd-4daf-b009-db840bedb933 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:25.468696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7b4c4381-05cd-4daf-b009-db840bedb933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:25.574575 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 8/13] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:58:25.697064 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec236a96-8a52-477d-8a56-5bcb30f98fd9 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:25.721688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:25.803325 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec236a96-8a52-477d-8a56-5bcb30f98fd9 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:25.836931 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512069d1-1c67-4ffa-81cc-58bbdacb4b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:25.837596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512069d1-1c67-4ffa-81cc-58bbdacb4b6d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:25.837875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512069d1-1c67-4ffa-81cc-58bbdacb4b6d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:25.838151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512069d1-1c67-4ffa-81cc-58bbdacb4b6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:25.840316 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c4381-05cd-4daf-b009-db840bedb933 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:25.849396 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512069d1-1c67-4ffa-81cc-58bbdacb4b6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:25.858121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-512069d1-1c67-4ffa-81cc-58bbdacb4b6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:58:25.858121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:25.858121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:25.858121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:25.858121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:25.858121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:25.858121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:25.858121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:25.858121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:25.858121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:25.858121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:25.858121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:25.858121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:25.858121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:25.858121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:25.858121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:25.859178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:25.859178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:25.859178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:25.859178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:25.859178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:25.859178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:25.859178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:25.859178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:25.859178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:25.859178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:25.859178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:25.859178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:25.859178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:25.859178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:25.859178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:25.859178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:25.860021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:25.860021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:25.860021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:25.860021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:25.860021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:25.860021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:25.860021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:25.860021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:25.860021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:25.860021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:25.860021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:58:25.860021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:58:25.860021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 6/14] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:58:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:25.866672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512069d1-1c67-4ffa-81cc-58bbdacb4b6d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:25.948536 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:25.962337 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c4381-05cd-4daf-b009-db840bedb933 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:25.964696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:25.966959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512069d1-1c67-4ffa-81cc-58bbdacb4b6d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:26.000602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d5951-35ae-451a-922b-3f546b7f1260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:26.001066 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d668722-ba49-4e86-9bad-858b637ab211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:26.001493 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d5951-35ae-451a-922b-3f546b7f1260 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:26.001707 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d668722-ba49-4e86-9bad-858b637ab211 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:26.001812 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d5951-35ae-451a-922b-3f546b7f1260 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:26.001985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d668722-ba49-4e86-9bad-858b637ab211 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:26.002071 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d5951-35ae-451a-922b-3f546b7f1260 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:26.002256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d668722-ba49-4e86-9bad-858b637ab211 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:26.002972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d668722-ba49-4e86-9bad-858b637ab211 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:26.003425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d668722-ba49-4e86-9bad-858b637ab211 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:26.004841 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d668722-ba49-4e86-9bad-858b637ab211 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:26.017762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 7/15] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:26.023614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d668722-ba49-4e86-9bad-858b637ab211 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:26.068935 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d5951-35ae-451a-922b-3f546b7f1260 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:26.070474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772d5951-35ae-451a-922b-3f546b7f1260 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:26.122103 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d668722-ba49-4e86-9bad-858b637ab211 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:26.155431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37825ffb-6451-451c-9297-4e92b35482d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:26.156047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37825ffb-6451-451c-9297-4e92b35482d0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:26.156323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37825ffb-6451-451c-9297-4e92b35482d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:26.156592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37825ffb-6451-451c-9297-4e92b35482d0 None admin] PATH_INFO: `/v3/users/2d817dbb81e84447ac0c66c6621a6e8f` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:26.157753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37825ffb-6451-451c-9297-4e92b35482d0 None admin] RBAC: Authorizing `identity:get_user(user_id=2d817dbb81e84447ac0c66c6621a6e8f)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:26.159194 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37825ffb-6451-451c-9297-4e92b35482d0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:26.160752 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 8/16] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:26 2022] GET /identity/v3/users/2d817dbb81e84447ac0c66c6621a6e8f => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:26.166291 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37825ffb-6451-451c-9297-4e92b35482d0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:26.167004 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 9/17] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:58:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 18:58:26.172263 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d5951-35ae-451a-922b-3f546b7f1260 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:26.175038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:26.180983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:26.267732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37825ffb-6451-451c-9297-4e92b35482d0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:26.271562 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d5951-35ae-451a-922b-3f546b7f1260 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:26.301070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d595167-22f7-4eee-8e3b-2ea6cf267a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:26.301717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d595167-22f7-4eee-8e3b-2ea6cf267a0d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:26.301994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d595167-22f7-4eee-8e3b-2ea6cf267a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:26.302267 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d595167-22f7-4eee-8e3b-2ea6cf267a0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:26.303370 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d595167-22f7-4eee-8e3b-2ea6cf267a0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:26.304258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790955d-4970-463c-b634-01185834fde7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:26.304804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d595167-22f7-4eee-8e3b-2ea6cf267a0d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:26.305090 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790955d-4970-463c-b634-01185834fde7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:26.305392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790955d-4970-463c-b634-01185834fde7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:26.305659 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f790955d-4970-463c-b634-01185834fde7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:26.306379 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 9/18] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:58:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:26.312666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d595167-22f7-4eee-8e3b-2ea6cf267a0d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:26.324465 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790955d-4970-463c-b634-01185834fde7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:26.326301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f790955d-4970-463c-b634-01185834fde7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:26.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-f790955d-4970-463c-b634-01185834fde7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:58:26.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:26.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:26.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:26.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:26.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:26.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:26.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:26.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:26.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:26.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:26.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:26.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:58:26.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:58:26.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:58:26.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:26.345052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:58:26.345888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:58:26.345888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:58:26.345888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:58:26.345888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 10/19] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:58:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 18:58:26.351062 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790955d-4970-463c-b634-01185834fde7 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:26.393057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:26.407100 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:26.412557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d595167-22f7-4eee-8e3b-2ea6cf267a0d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:26.445408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0951da-decb-4414-8148-d4c9df03d358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:26.446003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0951da-decb-4414-8148-d4c9df03d358 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:26.446275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0951da-decb-4414-8148-d4c9df03d358 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:26.446540 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0951da-decb-4414-8148-d4c9df03d358 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:26.447248 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0951da-decb-4414-8148-d4c9df03d358 None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.domain.id=default, user.id=2d817dbb81e84447ac0c66c6621a6e8f) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:26.447546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0951da-decb-4414-8148-d4c9df03d358 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:26.448959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0951da-decb-4414-8148-d4c9df03d358 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:26.452281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f790955d-4970-463c-b634-01185834fde7 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:26.455358 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 10/20] 10.222.0.65 () {56 vars in 1322 bytes} [Wed Aug 31 18:58:26 2022] GET /identity/v3/role_assignments?user.id=2d817dbb81e84447ac0c66c6621a6e8f&scope.domain.id=default&role.id=3580c894dd974d35bc95848b8ba22598 => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:26.488063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3863120-d7c0-4537-933d-fd7832b4e0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:26.488688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3863120-d7c0-4537-933d-fd7832b4e0cb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:26.488969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3863120-d7c0-4537-933d-fd7832b4e0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:26.489273 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3863120-d7c0-4537-933d-fd7832b4e0cb None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:26.489935 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3863120-d7c0-4537-933d-fd7832b4e0cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:26.490237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3863120-d7c0-4537-933d-fd7832b4e0cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:26.491645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3863120-d7c0-4537-933d-fd7832b4e0cb None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:26.512181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 11/21] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:28.464555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c020fe6-66e0-4e74-a50e-5c42f732d6eb None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:28.465034 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c020fe6-66e0-4e74-a50e-5c42f732d6eb None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:28.465034 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c020fe6-66e0-4e74-a50e-5c42f732d6eb None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:28.466675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 11/22] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:28.472208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991769dc-fd13-4641-966f-77d904bdaa2c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:28.472618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991769dc-fd13-4641-966f-77d904bdaa2c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:28.472809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991769dc-fd13-4641-966f-77d904bdaa2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:28.498181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-991769dc-fd13-4641-966f-77d904bdaa2c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:28.499384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-991769dc-fd13-4641-966f-77d904bdaa2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:28.582488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 12/23] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:28.758762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e6ec8-b5cf-4555-af4e-c1ab75d50b6e None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:28.759030 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e6ec8-b5cf-4555-af4e-c1ab75d50b6e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:28.759313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e6ec8-b5cf-4555-af4e-c1ab75d50b6e None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:28.761531 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 12/24] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:28.766203 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858405f-7284-4de8-befb-657e6c9e0c90 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:28.766543 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858405f-7284-4de8-befb-657e6c9e0c90 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:28.766817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1858405f-7284-4de8-befb-657e6c9e0c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:28.793378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-1858405f-7284-4de8-befb-657e6c9e0c90 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:28.794543 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1858405f-7284-4de8-befb-657e6c9e0c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:28.857837 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:28.890901 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18485100-0b01-4f39-bbaf-6dd06bf690cf None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:28.891233 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18485100-0b01-4f39-bbaf-6dd06bf690cf None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:28.891461 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18485100-0b01-4f39-bbaf-6dd06bf690cf None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:28.891565 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 13/25] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:28.898934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 13/26] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:28 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:28.903886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a6dc7-78b0-4cd1-b894-4eee72f68513 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:28.904174 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a6dc7-78b0-4cd1-b894-4eee72f68513 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:28.904442 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a6dc7-78b0-4cd1-b894-4eee72f68513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:28.930206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-175a6dc7-78b0-4cd1-b894-4eee72f68513 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:28.931396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-175a6dc7-78b0-4cd1-b894-4eee72f68513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:29.024032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 14/27] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:29.180279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18485100-0b01-4f39-bbaf-6dd06bf690cf None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:29.302887 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a6dc7-78b0-4cd1-b894-4eee72f68513 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:29.303174 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18485100-0b01-4f39-bbaf-6dd06bf690cf None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:29.363019 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35384c56-d069-41a8-b1b7-162c133b423c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:29.363882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35384c56-d069-41a8-b1b7-162c133b423c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:29.364331 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35384c56-d069-41a8-b1b7-162c133b423c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:29.364745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35384c56-d069-41a8-b1b7-162c133b423c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:29.390109 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35384c56-d069-41a8-b1b7-162c133b423c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:29.395938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-35384c56-d069-41a8-b1b7-162c133b423c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:58:29.395938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:29.395938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:29.395938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:29.395938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:29.395938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:29.395938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:29.395938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:29.395938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:29.395938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:29.395938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:29.395938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:29.395938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:29.395938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:29.395938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:29.395938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:29.396872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:29.396872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:29.396872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:29.396872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:29.396872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:29.396872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:29.396872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:29.396872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:29.396872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:29.396872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:29.396872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:29.396872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:29.396872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:29.396872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:29.396872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:29.396872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:29.397684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:29.397684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:29.397684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:29.397684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:29.397684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:29.397684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:29.397684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:29.397684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:29.397684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:29.397684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:29.397684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:58:29.397684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:58:29.406769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 14/28] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:58:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:29.412987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35384c56-d069-41a8-b1b7-162c133b423c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:29.426595 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:29.476174 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a6dc7-78b0-4cd1-b894-4eee72f68513 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:29.485559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:29.539141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b90e71-80be-44b3-b757-b127302a632a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:29.539770 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b90e71-80be-44b3-b757-b127302a632a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:29.540048 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b90e71-80be-44b3-b757-b127302a632a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:29.540316 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b90e71-80be-44b3-b757-b127302a632a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:29.558880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b90e71-80be-44b3-b757-b127302a632a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:29.560269 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b90e71-80be-44b3-b757-b127302a632a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:29.565732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35384c56-d069-41a8-b1b7-162c133b423c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:29.589962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-61b90e71-80be-44b3-b757-b127302a632a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:58:29.589962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:29.589962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:29.589962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:29.589962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:29.589962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:29.589962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:29.589962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:29.589962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:29.589962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:29.589962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:29.589962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:29.589962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:58:29.589962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:58:29.589962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:58:29.589962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:29.590861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:58:29.591629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:58:29.591629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:58:29.591629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:58:29.593450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 15/29] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:58:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 18:58:29.594357 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b90e71-80be-44b3-b757-b127302a632a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:29.630612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959edc3-8fdc-4c80-b896-7c760ff1df6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:29.631538 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959edc3-8fdc-4c80-b896-7c760ff1df6d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:29.632001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959edc3-8fdc-4c80-b896-7c760ff1df6d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:29.632446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6959edc3-8fdc-4c80-b896-7c760ff1df6d None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:29.633351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959edc3-8fdc-4c80-b896-7c760ff1df6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:29.633834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959edc3-8fdc-4c80-b896-7c760ff1df6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:29.635415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6959edc3-8fdc-4c80-b896-7c760ff1df6d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:29.652638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 15/30] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:29.653883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959edc3-8fdc-4c80-b896-7c760ff1df6d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:29.777118 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:29.781872 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b90e71-80be-44b3-b757-b127302a632a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:29.832184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6959edc3-8fdc-4c80-b896-7c760ff1df6d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:29.836839 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929c14b0-cd62-4967-b9ab-2ab5ed995f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:29.837555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c14b0-cd62-4967-b9ab-2ab5ed995f6b None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:29.837907 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c14b0-cd62-4967-b9ab-2ab5ed995f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:29.838792 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929c14b0-cd62-4967-b9ab-2ab5ed995f6b None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:29.839527 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c14b0-cd62-4967-b9ab-2ab5ed995f6b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:29.840978 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929c14b0-cd62-4967-b9ab-2ab5ed995f6b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:29.871591 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe914f2-bc5c-4023-9e2f-2f5db71521b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:29.872785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe914f2-bc5c-4023-9e2f-2f5db71521b2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:29.873263 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe914f2-bc5c-4023-9e2f-2f5db71521b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:29.873712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe914f2-bc5c-4023-9e2f-2f5db71521b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:29.874593 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe914f2-bc5c-4023-9e2f-2f5db71521b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:29.875117 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe914f2-bc5c-4023-9e2f-2f5db71521b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:29.876693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe914f2-bc5c-4023-9e2f-2f5db71521b2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:29.899914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 16/31] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:58:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:29.901269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe914f2-bc5c-4023-9e2f-2f5db71521b2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:29.971524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 16/32] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 31 18:58:29 2022] POST /identity/v3/roles => generated 219 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:58:29.973341 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee658f1-930a-4741-ac0a-78a9791f87ba None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:29.973780 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee658f1-930a-4741-ac0a-78a9791f87ba None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:29.974207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee658f1-930a-4741-ac0a-78a9791f87ba None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:29.975968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 17/33] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:29.976927 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee658f1-930a-4741-ac0a-78a9791f87ba None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:30.004177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:30.060053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe914f2-bc5c-4023-9e2f-2f5db71521b2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:30.113712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:30.126693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddac1f74-bd77-4b1f-9ec1-e608e79f9dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:30.127644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac1f74-bd77-4b1f-9ec1-e608e79f9dff None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:30.128122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac1f74-bd77-4b1f-9ec1-e608e79f9dff None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:30.128576 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddac1f74-bd77-4b1f-9ec1-e608e79f9dff None admin] PATH_INFO: `/v3/users/2d817dbb81e84447ac0c66c6621a6e8f` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:30.131300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac1f74-bd77-4b1f-9ec1-e608e79f9dff None admin] RBAC: Authorizing `identity:get_user(user_id=2d817dbb81e84447ac0c66c6621a6e8f)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:30.132991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddac1f74-bd77-4b1f-9ec1-e608e79f9dff None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:30.138331 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 17/34] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:29 2022] GET /identity/v3/users/2d817dbb81e84447ac0c66c6621a6e8f => generated 247 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:30.139857 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26701096-0c49-4002-ac73-ea41ba71c413 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:30.141153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26701096-0c49-4002-ac73-ea41ba71c413 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:30.142422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26701096-0c49-4002-ac73-ea41ba71c413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:30.170022 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee658f1-930a-4741-ac0a-78a9791f87ba None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:30.170968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-26701096-0c49-4002-ac73-ea41ba71c413 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:30.172459 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-26701096-0c49-4002-ac73-ea41ba71c413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:30.216312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26720614-f856-4b2d-bbe4-11d3a19dd333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:30.216939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26720614-f856-4b2d-bbe4-11d3a19dd333 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:30.217298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26720614-f856-4b2d-bbe4-11d3a19dd333 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:30.217576 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26720614-f856-4b2d-bbe4-11d3a19dd333 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:30.218620 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26720614-f856-4b2d-bbe4-11d3a19dd333 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:30.220215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26720614-f856-4b2d-bbe4-11d3a19dd333 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:30.262458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 18/35] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 18:58:29 2022] POST /identity/v3/projects => generated 301 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:58:30.263332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26720614-f856-4b2d-bbe4-11d3a19dd333 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:30.265314 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 18/36] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:30.277778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d23343-6e92-47fe-b927-fee679b59de3 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:30.278406 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d23343-6e92-47fe-b927-fee679b59de3 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:30.278928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d23343-6e92-47fe-b927-fee679b59de3 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:30.282474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 19/37] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:30.290402 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a4cdf-4dcb-465c-ad8b-aeafda007c5a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:30.291013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a4cdf-4dcb-465c-ad8b-aeafda007c5a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:30.291611 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a4cdf-4dcb-465c-ad8b-aeafda007c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:30.325646 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-bb2a4cdf-4dcb-465c-ad8b-aeafda007c5a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:30.327275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bb2a4cdf-4dcb-465c-ad8b-aeafda007c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:30.356284 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:30.418111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 20/38] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:30.419431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26720614-f856-4b2d-bbe4-11d3a19dd333 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:30.475022 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94042464-0a63-4ce2-bae9-c315d9ad6760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:30.475673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94042464-0a63-4ce2-bae9-c315d9ad6760 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:30.475957 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94042464-0a63-4ce2-bae9-c315d9ad6760 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:30.476235 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94042464-0a63-4ce2-bae9-c315d9ad6760 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:30.477410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94042464-0a63-4ce2-bae9-c315d9ad6760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:30.478905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94042464-0a63-4ce2-bae9-c315d9ad6760 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:30.480499 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 19/39] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:58:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:30.486475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a4cdf-4dcb-465c-ad8b-aeafda007c5a None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:30.495853 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581a4b1-bb39-4690-83c9-ce246c0a5eaf None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:30.496136 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581a4b1-bb39-4690-83c9-ce246c0a5eaf None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:30.496411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8581a4b1-bb39-4690-83c9-ce246c0a5eaf None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:30.498024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 20/40] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 18:58:30.503311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba34151-94ad-430f-b768-13dc87fa924e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:30.503598 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba34151-94ad-430f-b768-13dc87fa924e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:30.503875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba34151-94ad-430f-b768-13dc87fa924e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:30.531264 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2ba34151-94ad-430f-b768-13dc87fa924e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:30.531264 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2ba34151-94ad-430f-b768-13dc87fa924e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:30.556290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:30.610086 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 21/41] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:58:30.610747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba34151-94ad-430f-b768-13dc87fa924e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:30.617143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a4cdf-4dcb-465c-ad8b-aeafda007c5a None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:30.653959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb264230-7d4f-49da-b8f9-46e8e54dd9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:30.654670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb264230-7d4f-49da-b8f9-46e8e54dd9f8 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:30.654924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb264230-7d4f-49da-b8f9-46e8e54dd9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:30.655208 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb264230-7d4f-49da-b8f9-46e8e54dd9f8 None admin] PATH_INFO: `/v3/domains/default/users/2d817dbb81e84447ac0c66c6621a6e8f/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:30.657353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb264230-7d4f-49da-b8f9-46e8e54dd9f8 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=2d817dbb81e84447ac0c66c6621a6e8f, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:30.659457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb264230-7d4f-49da-b8f9-46e8e54dd9f8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:30.695950 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:30.736738 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba34151-94ad-430f-b768-13dc87fa924e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:30.772774 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717490d-85f7-4967-8f06-729405bc408f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:30.773456 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717490d-85f7-4967-8f06-729405bc408f None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:30.773736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717490d-85f7-4967-8f06-729405bc408f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:30.774002 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717490d-85f7-4967-8f06-729405bc408f None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:30.774657 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717490d-85f7-4967-8f06-729405bc408f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:30.776052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8717490d-85f7-4967-8f06-729405bc408f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:30.788045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 21/42] 10.222.0.65 () {58 vars in 1361 bytes} [Wed Aug 31 18:58:30 2022] PUT /identity/v3/domains/default/users/2d817dbb81e84447ac0c66c6621a6e8f/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 18:58:30.789079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb264230-7d4f-49da-b8f9-46e8e54dd9f8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:30.795174 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 22/43] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 31 18:58:30 2022] POST /identity/v3/roles => generated 225 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:58:30.885871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717490d-85f7-4967-8f06-729405bc408f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:31.017425 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717490d-85f7-4967-8f06-729405bc408f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:31.042388 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb264230-7d4f-49da-b8f9-46e8e54dd9f8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:31.050666 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:31.057910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:31.062197 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a20b6a-2b1e-4902-a3f8-7b06db2628ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:31.063149 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a20b6a-2b1e-4902-a3f8-7b06db2628ce None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:31.063462 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a20b6a-2b1e-4902-a3f8-7b06db2628ce None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:31.063767 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a20b6a-2b1e-4902-a3f8-7b06db2628ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:31.064937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a20b6a-2b1e-4902-a3f8-7b06db2628ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:31.066892 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a20b6a-2b1e-4902-a3f8-7b06db2628ce None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:31.068641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 23/44] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:58:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:31.074121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a20b6a-2b1e-4902-a3f8-7b06db2628ce None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:31.080173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ab2b1a-5b70-4387-a7ed-ba1dffd5982e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:31.080807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab2b1a-5b70-4387-a7ed-ba1dffd5982e None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:31.081113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab2b1a-5b70-4387-a7ed-ba1dffd5982e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:31.081410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ab2b1a-5b70-4387-a7ed-ba1dffd5982e None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:31.082073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab2b1a-5b70-4387-a7ed-ba1dffd5982e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:31.083650 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04ab2b1a-5b70-4387-a7ed-ba1dffd5982e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:31.103352 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 22/45] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 31 18:58:30 2022] POST /identity/v3/roles => generated 223 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:58:31.188386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a20b6a-2b1e-4902-a3f8-7b06db2628ce None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:31.231761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ec135-593f-4044-b89f-9fa7398ee868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:31.232388 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ec135-593f-4044-b89f-9fa7398ee868 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:31.232670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ec135-593f-4044-b89f-9fa7398ee868 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:31.232938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ec135-593f-4044-b89f-9fa7398ee868 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:31.233622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ec135-593f-4044-b89f-9fa7398ee868 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:31.235235 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ec135-593f-4044-b89f-9fa7398ee868 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:31.266205 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 24/46] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 18:58:31 2022] POST /identity/v3/projects => generated 312 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 18:58:32.737169 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62350d93-0fc9-404d-9b45-ec7fdfd540f1 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:32.737743 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62350d93-0fc9-404d-9b45-ec7fdfd540f1 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:32.737743 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62350d93-0fc9-404d-9b45-ec7fdfd540f1 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:32.739090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 23/47] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:32.744389 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e72310-fb5a-45be-b62d-7a9143679c6c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:32.745381 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e72310-fb5a-45be-b62d-7a9143679c6c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:32.745603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e72310-fb5a-45be-b62d-7a9143679c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:32.764090 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d8e72310-fb5a-45be-b62d-7a9143679c6c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:32.765303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d8e72310-fb5a-45be-b62d-7a9143679c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:32.807773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:32.831590 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 25/48] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:33.102475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62350d93-0fc9-404d-9b45-ec7fdfd540f1 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:33.153064 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3e34e-163c-46a1-b489-6b30b90ed997 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:33.153372 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3e34e-163c-46a1-b489-6b30b90ed997 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:33.153648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3e34e-163c-46a1-b489-6b30b90ed997 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:33.154975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 26/49] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 18:58:33.159738 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6db3a-79d7-41b9-92f7-6caf917e58be None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:33.160018 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6db3a-79d7-41b9-92f7-6caf917e58be None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:33.160285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad6db3a-79d7-41b9-92f7-6caf917e58be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:33.178175 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-1ad6db3a-79d7-41b9-92f7-6caf917e58be None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:33.179351 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1ad6db3a-79d7-41b9-92f7-6caf917e58be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:33.207559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62350d93-0fc9-404d-9b45-ec7fdfd540f1 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:33.246092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4985af0-d7e0-44de-b74c-77e5106b5fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:33.246745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4985af0-d7e0-44de-b74c-77e5106b5fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:33.247273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4985af0-d7e0-44de-b74c-77e5106b5fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:33.247542 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4985af0-d7e0-44de-b74c-77e5106b5fa8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:33.259435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4985af0-d7e0-44de-b74c-77e5106b5fa8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:33.264977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-a4985af0-d7e0-44de-b74c-77e5106b5fa8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:58:33.264977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:33.264977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:33.264977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:33.264977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:33.264977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:33.264977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:33.264977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:33.264977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:33.264977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:33.264977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:33.264977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:33.264977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:33.264977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:33.264977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:33.264977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:33.265915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:33.265915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:33.265915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:33.265915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:33.265915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:33.265915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:33.265915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:33.265915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:33.265915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:33.265915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:33.265915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:33.265915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:33.265915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:33.265915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:33.265915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:33.265915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:33.266656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:33.266656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:33.266656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:33.266656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:33.266656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:33.266656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:33.266656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:33.266656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:33.266656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:33.266656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:33.266656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:58:33.266656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:58:33.266656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 24/50] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:58:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:33.272956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4985af0-d7e0-44de-b74c-77e5106b5fa8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:33.310061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:33.350928 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 27/51] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:58:33.378168 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4985af0-d7e0-44de-b74c-77e5106b5fa8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:33.411193 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984af8f-121c-44b6-aa8b-0a2099dd6e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:33.411791 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984af8f-121c-44b6-aa8b-0a2099dd6e47 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:33.412065 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984af8f-121c-44b6-aa8b-0a2099dd6e47 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:33.412333 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b984af8f-121c-44b6-aa8b-0a2099dd6e47 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:33.412983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984af8f-121c-44b6-aa8b-0a2099dd6e47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:33.413309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984af8f-121c-44b6-aa8b-0a2099dd6e47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:33.414681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b984af8f-121c-44b6-aa8b-0a2099dd6e47 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:33.422981 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 25/52] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:33.428524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6db3a-79d7-41b9-92f7-6caf917e58be None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:33.523815 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad6db3a-79d7-41b9-92f7-6caf917e58be None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:33.535052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:33.560481 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf397c0-aeca-4067-9f08-ed48c8bdf063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:33.561145 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf397c0-aeca-4067-9f08-ed48c8bdf063 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:33.561459 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf397c0-aeca-4067-9f08-ed48c8bdf063 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:33.561737 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf397c0-aeca-4067-9f08-ed48c8bdf063 None admin] PATH_INFO: `/v3/users/2d817dbb81e84447ac0c66c6621a6e8f` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:33.563080 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf397c0-aeca-4067-9f08-ed48c8bdf063 None admin] RBAC: Authorizing `identity:get_user(user_id=2d817dbb81e84447ac0c66c6621a6e8f)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:33.564531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf397c0-aeca-4067-9f08-ed48c8bdf063 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:33.566154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 28/53] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:33 2022] GET /identity/v3/users/2d817dbb81e84447ac0c66c6621a6e8f => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:33.571853 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984af8f-121c-44b6-aa8b-0a2099dd6e47 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:33.620291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf397c0-aeca-4067-9f08-ed48c8bdf063 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:33.669029 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:33.675495 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b984af8f-121c-44b6-aa8b-0a2099dd6e47 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:33.710693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f52377-fc8f-46b8-801f-2cddebaf0cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:33.711340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f52377-fc8f-46b8-801f-2cddebaf0cdb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:33.711613 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f52377-fc8f-46b8-801f-2cddebaf0cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:33.711884 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f52377-fc8f-46b8-801f-2cddebaf0cdb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:33.713063 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f52377-fc8f-46b8-801f-2cddebaf0cdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:33.715085 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f52377-fc8f-46b8-801f-2cddebaf0cdb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:33.716648 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 26/54] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:58:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:33.720090 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf397c0-aeca-4067-9f08-ed48c8bdf063 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:33.723149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f52377-fc8f-46b8-801f-2cddebaf0cdb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:33.753542 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da4f80-a0dd-484c-bc3b-99b06a006e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:33.754166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da4f80-a0dd-484c-bc3b-99b06a006e36 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:33.754446 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da4f80-a0dd-484c-bc3b-99b06a006e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:33.754722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6da4f80-a0dd-484c-bc3b-99b06a006e36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:33.755845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da4f80-a0dd-484c-bc3b-99b06a006e36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:33.757306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6da4f80-a0dd-484c-bc3b-99b06a006e36 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:33.758879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 29/55] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:58:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:33.764761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da4f80-a0dd-484c-bc3b-99b06a006e36 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:33.808997 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:33.821972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f52377-fc8f-46b8-801f-2cddebaf0cdb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:33.855372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd019567-44c5-4af3-b1d9-8968f84e75c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:33.855967 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd019567-44c5-4af3-b1d9-8968f84e75c0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:33.856242 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd019567-44c5-4af3-b1d9-8968f84e75c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:33.856512 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd019567-44c5-4af3-b1d9-8968f84e75c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:33.857242 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd019567-44c5-4af3-b1d9-8968f84e75c0 None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.domain.id=default, user.id=2d817dbb81e84447ac0c66c6621a6e8f) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:33.857555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd019567-44c5-4af3-b1d9-8968f84e75c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:33.858963 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd019567-44c5-4af3-b1d9-8968f84e75c0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:33.862732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6da4f80-a0dd-484c-bc3b-99b06a006e36 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:33.864625 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 27/56] 10.222.0.65 () {56 vars in 1322 bytes} [Wed Aug 31 18:58:33 2022] GET /identity/v3/role_assignments?user.id=2d817dbb81e84447ac0c66c6621a6e8f&scope.domain.id=default&role.id=3580c894dd974d35bc95848b8ba22598 => generated 530 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:33.898381 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc3e73a-c765-4ed1-8685-8a2cdead9f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:33.899003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3e73a-c765-4ed1-8685-8a2cdead9f5d None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:33.899285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3e73a-c765-4ed1-8685-8a2cdead9f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:33.899557 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc3e73a-c765-4ed1-8685-8a2cdead9f5d None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:33.900218 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3e73a-c765-4ed1-8685-8a2cdead9f5d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:33.901813 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc3e73a-c765-4ed1-8685-8a2cdead9f5d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:33.930349 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 30/57] 10.222.0.65 () {60 vars in 1140 bytes} [Wed Aug 31 18:58:33 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 18:58:35.810989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061f287-0a91-43c2-806d-b81af009e27e None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:35.811530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061f287-0a91-43c2-806d-b81af009e27e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:35.811530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d061f287-0a91-43c2-806d-b81af009e27e None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:35.812879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 28/58] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:35.818024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe7491-0713-4ee8-b364-5980d3dee9f4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:35.818427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe7491-0713-4ee8-b364-5980d3dee9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:35.818580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbe7491-0713-4ee8-b364-5980d3dee9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:35.836807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ccbe7491-0713-4ee8-b364-5980d3dee9f4 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:35.838035 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ccbe7491-0713-4ee8-b364-5980d3dee9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:35.901866 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 31/59] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:36.171826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061f287-0a91-43c2-806d-b81af009e27e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:36.199954 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:36.271235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d061f287-0a91-43c2-806d-b81af009e27e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:36.303219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768a25c6-0f1d-4327-a6fa-facbccec0558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:36.303827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a25c6-0f1d-4327-a6fa-facbccec0558 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:36.304102 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a25c6-0f1d-4327-a6fa-facbccec0558 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:36.304370 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768a25c6-0f1d-4327-a6fa-facbccec0558 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:36.305470 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a25c6-0f1d-4327-a6fa-facbccec0558 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:36.306879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768a25c6-0f1d-4327-a6fa-facbccec0558 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:36.309005 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 29/60] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:58:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:36.314472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe7491-0713-4ee8-b364-5980d3dee9f4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:36.340672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:36.411774 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbe7491-0713-4ee8-b364-5980d3dee9f4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:36.444971 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0d59b4-6fe6-4b9e-8c36-ae5797f3959f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:36.445533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d59b4-6fe6-4b9e-8c36-ae5797f3959f None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:36.445815 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d59b4-6fe6-4b9e-8c36-ae5797f3959f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:36.446088 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0d59b4-6fe6-4b9e-8c36-ae5797f3959f None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:36.446730 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d59b4-6fe6-4b9e-8c36-ae5797f3959f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:36.448204 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0d59b4-6fe6-4b9e-8c36-ae5797f3959f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:36.473950 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 32/61] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 18:58:36 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:58:39.311499 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80255327-8d93-4948-85d6-9fb0805f54c8 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:39.312000 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80255327-8d93-4948-85d6-9fb0805f54c8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:39.312073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80255327-8d93-4948-85d6-9fb0805f54c8 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:39.313728 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 30/62] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:39.318955 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8189be1-7050-4aed-9c87-a20211184d04 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:39.319353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8189be1-7050-4aed-9c87-a20211184d04 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:39.319514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8189be1-7050-4aed-9c87-a20211184d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:39.339191 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d8189be1-7050-4aed-9c87-a20211184d04 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:39.340693 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d8189be1-7050-4aed-9c87-a20211184d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:39.423786 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 33/63] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:39.567004 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cde55-1564-44a1-a1d2-56c8797fe8bc None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:39.567287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cde55-1564-44a1-a1d2-56c8797fe8bc None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:39.567722 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9cde55-1564-44a1-a1d2-56c8797fe8bc None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:39.569456 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 31/64] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:39.575385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bbe10-fbc3-456a-a6eb-ad759fddb377 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:39.575914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bbe10-fbc3-456a-a6eb-ad759fddb377 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:39.576383 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8bbe10-fbc3-456a-a6eb-ad759fddb377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:39.594589 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8f8bbe10-fbc3-456a-a6eb-ad759fddb377 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:39.596056 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8f8bbe10-fbc3-456a-a6eb-ad759fddb377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:39.603903 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:39.678763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 34/65] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:39.776875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cde55-1564-44a1-a1d2-56c8797fe8bc None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:39.869026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17486de-56b6-460e-8c1e-bc4ca15f0974 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:39.870493 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17486de-56b6-460e-8c1e-bc4ca15f0974 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:39.871239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17486de-56b6-460e-8c1e-bc4ca15f0974 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:39.873058 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 35/66] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 18:58:39.878572 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ef278-1578-4644-afcf-ce0492ed5a1b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:39.879441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ef278-1578-4644-afcf-ce0492ed5a1b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:39.879920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53ef278-1578-4644-afcf-ce0492ed5a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:39.902198 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f53ef278-1578-4644-afcf-ce0492ed5a1b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:39.904397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f53ef278-1578-4644-afcf-ce0492ed5a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:39.914853 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:39.924457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9cde55-1564-44a1-a1d2-56c8797fe8bc None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:39.966900 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f9f19-20f9-4e3a-87af-ff86a3cfc4b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:39.967541 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f9f19-20f9-4e3a-87af-ff86a3cfc4b8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:39.967826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f9f19-20f9-4e3a-87af-ff86a3cfc4b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:39.968105 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f9f19-20f9-4e3a-87af-ff86a3cfc4b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:39.982512 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f9f19-20f9-4e3a-87af-ff86a3cfc4b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:39.989202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-e19f9f19-20f9-4e3a-87af-ff86a3cfc4b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:58:39.989202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:39.989202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:39.989202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:39.989202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:39.989202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:39.989202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:39.989202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:39.989202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:39.989202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:39.989202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:39.989202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:39.989202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:39.989202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:39.989202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:39.989202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:39.990167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:39.990167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:39.990167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:39.990167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:39.990167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:39.990167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:39.990167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:39.990167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:39.990167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:39.990167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:39.990167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:39.990167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:39.990167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:39.990167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:39.990167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:39.990167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:39.990917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:39.990917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:39.990917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:39.990917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:39.990917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:39.990917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:39.990917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:39.990917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:39.990917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:39.990917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:39.990917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:58:39.990917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:58:39.993130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 32/67] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:39.993896 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 36/68] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:39.994685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ee176-ec73-49a4-a1c8-a1b54ef077ff None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:39.995529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ee176-ec73-49a4-a1c8-a1b54ef077ff None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:39.995952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ee176-ec73-49a4-a1c8-a1b54ef077ff None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:40.002745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ef278-1578-4644-afcf-ce0492ed5a1b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:40.002917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 33/69] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:40.008442 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c0f50-3f5d-464d-a932-448fade48274 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:40.009989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c0f50-3f5d-464d-a932-448fade48274 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:40.010468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5c0f50-3f5d-464d-a932-448fade48274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:40.049433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4a5c0f50-3f5d-464d-a932-448fade48274 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:40.052136 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4a5c0f50-3f5d-464d-a932-448fade48274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:40.119779 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:40.151599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53ef278-1578-4644-afcf-ce0492ed5a1b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:40.158931 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 34/70] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:40.160536 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c0f50-3f5d-464d-a932-448fade48274 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:40.216776 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd508d97-b595-46d5-9f09-d0ae9b53e1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:40.218608 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd508d97-b595-46d5-9f09-d0ae9b53e1bf None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:40.219737 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd508d97-b595-46d5-9f09-d0ae9b53e1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:40.220311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd508d97-b595-46d5-9f09-d0ae9b53e1bf None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:40.222250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd508d97-b595-46d5-9f09-d0ae9b53e1bf None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:40.222702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd508d97-b595-46d5-9f09-d0ae9b53e1bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:40.225356 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd508d97-b595-46d5-9f09-d0ae9b53e1bf None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:40.249027 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 37/71] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:58:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:40.255820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd508d97-b595-46d5-9f09-d0ae9b53e1bf None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:40.321914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:40.341753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5c0f50-3f5d-464d-a932-448fade48274 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:40.378694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd508d97-b595-46d5-9f09-d0ae9b53e1bf None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:40.380262 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245ef22-088e-42af-9e8e-7b2363e49fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:40.380895 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245ef22-088e-42af-9e8e-7b2363e49fdc None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:40.381175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245ef22-088e-42af-9e8e-7b2363e49fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:40.381475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3245ef22-088e-42af-9e8e-7b2363e49fdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:40.392023 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3245ef22-088e-42af-9e8e-7b2363e49fdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:40.396860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-3245ef22-088e-42af-9e8e-7b2363e49fdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:58:40.396860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:40.396860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:40.396860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:40.396860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:40.396860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:40.396860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:40.396860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:40.396860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:40.396860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:40.396860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:40.396860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:40.396860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:40.396860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:40.396860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:40.396860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:40.397706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:40.397706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:40.397706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:40.397706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:40.397706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:40.397706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:40.397706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:40.397706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:40.397706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:40.397706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:40.397706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:40.397706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:40.397706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:40.397706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:40.397706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:40.397706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:40.398850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:40.398850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:40.398850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:40.398850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:40.398850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:40.398850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:40.398850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:40.398850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:40.398850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:40.398850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:40.398850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:58:40.398850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:58:40.398850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 35/72] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:58:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:40.398850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245ef22-088e-42af-9e8e-7b2363e49fdc None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:40.419650 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fd129b-31d5-4baa-a7ac-025bfce6086a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:40.420284 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd129b-31d5-4baa-a7ac-025bfce6086a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:40.420566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd129b-31d5-4baa-a7ac-025bfce6086a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:40.420839 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd129b-31d5-4baa-a7ac-025bfce6086a None admin] PATH_INFO: `/v3/users/fd866208c70e43639bb105515ea2921e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:40.422932 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:40.437775 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd129b-31d5-4baa-a7ac-025bfce6086a None admin] RBAC: Authorizing `identity:get_user(user_id=fd866208c70e43639bb105515ea2921e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:40.439243 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fd129b-31d5-4baa-a7ac-025bfce6086a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:40.440916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 38/73] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:40 2022] GET /identity/v3/users/fd866208c70e43639bb105515ea2921e => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:40.442100 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffb51a-8225-47ad-b1d0-ae3de5be7f40 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:40.442391 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffb51a-8225-47ad-b1d0-ae3de5be7f40 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:40.442668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffb51a-8225-47ad-b1d0-ae3de5be7f40 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:40.443867 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 39/74] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:40.444563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffb51a-8225-47ad-b1d0-ae3de5be7f40 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:40.504532 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3245ef22-088e-42af-9e8e-7b2363e49fdc None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:40.538532 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec71c0-0f04-4651-9aa4-851ab56f10fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:40.539353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec71c0-0f04-4651-9aa4-851ab56f10fd None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:40.539639 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec71c0-0f04-4651-9aa4-851ab56f10fd None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:40.539911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec71c0-0f04-4651-9aa4-851ab56f10fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:40.544424 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffb51a-8225-47ad-b1d0-ae3de5be7f40 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:40.551644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec71c0-0f04-4651-9aa4-851ab56f10fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:40.556410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-76ec71c0-0f04-4651-9aa4-851ab56f10fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:58:40.556410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:40.556410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:40.556410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:40.556410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:40.556410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:40.556410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:40.556410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:40.556410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:40.556410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:40.556410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:40.556410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:40.556410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:40.556410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:40.556410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:40.556410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:40.558050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:40.558050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:40.558050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:40.558050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:40.558050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:40.558050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:40.558050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:40.558050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:40.558050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:40.558050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:40.558050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:40.558050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:40.558050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:40.558050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:40.558050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:40.558050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:40.558817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:40.558817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:40.558817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:40.558817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:40.558817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:40.558817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:40.558817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:40.558817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:40.558817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:40.558817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:40.558817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:58:40.558817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:58:40.558817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 36/75] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:40.559431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec71c0-0f04-4651-9aa4-851ab56f10fd None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:40.568086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:40.582147 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c954289-4545-4345-b867-9408240d751f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:40.582760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c954289-4545-4345-b867-9408240d751f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:40.583038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c954289-4545-4345-b867-9408240d751f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:40.583306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c954289-4545-4345-b867-9408240d751f None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:40.583962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c954289-4545-4345-b867-9408240d751f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:40.584260 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c954289-4545-4345-b867-9408240d751f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:40.585646 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c954289-4545-4345-b867-9408240d751f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:40.593952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 40/76] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:40.594742 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c954289-4545-4345-b867-9408240d751f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:40.648709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:40.660036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec71c0-0f04-4651-9aa4-851ab56f10fd None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:40.693501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c954289-4545-4345-b867-9408240d751f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:40.694666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798169cb-7596-4f41-8497-3bb5626ebd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:40.695295 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798169cb-7596-4f41-8497-3bb5626ebd21 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:40.695564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798169cb-7596-4f41-8497-3bb5626ebd21 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:40.695842 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798169cb-7596-4f41-8497-3bb5626ebd21 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:40.705718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798169cb-7596-4f41-8497-3bb5626ebd21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:40.710464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-798169cb-7596-4f41-8497-3bb5626ebd21 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:58:40.710464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:40.710464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:40.710464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:40.710464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:40.710464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:40.710464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:40.710464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:40.710464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:40.710464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:40.710464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:40.710464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:40.710464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:40.710464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:40.710464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:40.710464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:40.711364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:40.711364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:40.711364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:40.711364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:40.711364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:40.711364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:40.711364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:40.711364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:40.711364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:40.711364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:40.711364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:40.711364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:40.711364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:40.711364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:40.711364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:40.711364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:40.712114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:40.712114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:40.712114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:40.712114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:40.712114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:40.712114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:40.712114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:40.712114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:40.712114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:40.712114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:40.712114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:58:40.712114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:58:40.712114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 37/77] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 18:58:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:40.712781 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf84a3-3755-4323-b3bb-23abd0c4f9d8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:40.713730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf84a3-3755-4323-b3bb-23abd0c4f9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:40.713730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf84a3-3755-4323-b3bb-23abd0c4f9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:40.726698 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcfb02-49c8-4e9e-9980-ba883c043210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:40.727300 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcfb02-49c8-4e9e-9980-ba883c043210 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:40.727583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcfb02-49c8-4e9e-9980-ba883c043210 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:40.727854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcfb02-49c8-4e9e-9980-ba883c043210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:40.730891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-5cbf84a3-3755-4323-b3bb-23abd0c4f9d8 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:40.732088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5cbf84a3-3755-4323-b3bb-23abd0c4f9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:40.790102 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:40.794776 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcfb02-49c8-4e9e-9980-ba883c043210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:40.796238 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcfb02-49c8-4e9e-9980-ba883c043210 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:40.798398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 38/78] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:40.799036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf84a3-3755-4323-b3bb-23abd0c4f9d8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:40.856714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:40.912296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf84a3-3755-4323-b3bb-23abd0c4f9d8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:40.915127 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 41/79] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:58:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:58:40.915783 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcfb02-49c8-4e9e-9980-ba883c043210 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:40.952121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71670d2-4ea2-413a-8330-25a34f796bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:40.952728 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71670d2-4ea2-413a-8330-25a34f796bbc None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:40.953010 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71670d2-4ea2-413a-8330-25a34f796bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:40.953319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71670d2-4ea2-413a-8330-25a34f796bbc None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:40.953996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71670d2-4ea2-413a-8330-25a34f796bbc None admin] RBAC: Adding query filter params (name=member) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:40.954277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71670d2-4ea2-413a-8330-25a34f796bbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:40.955646 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71670d2-4ea2-413a-8330-25a34f796bbc None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:40.965663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 39/80] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:58:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:40.966296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71670d2-4ea2-413a-8330-25a34f796bbc None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:41.023184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcfb02-49c8-4e9e-9980-ba883c043210 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:41.029018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:41.058831 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3d830-d261-4b72-8199-165e51470eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:41.059489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3d830-d261-4b72-8199-165e51470eba None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:41.059769 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3d830-d261-4b72-8199-165e51470eba None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:41.060042 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f3d830-d261-4b72-8199-165e51470eba None admin] PATH_INFO: `/v3/users/2d817dbb81e84447ac0c66c6621a6e8f` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:41.061176 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3d830-d261-4b72-8199-165e51470eba None admin] RBAC: Authorizing `identity:get_user(user_id=2d817dbb81e84447ac0c66c6621a6e8f)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:41.062702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f3d830-d261-4b72-8199-165e51470eba None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:41.069024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 42/81] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:40 2022] GET /identity/v3/users/2d817dbb81e84447ac0c66c6621a6e8f => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:41.069024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3d830-d261-4b72-8199-165e51470eba None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:41.081182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71670d2-4ea2-413a-8330-25a34f796bbc None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:41.092596 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:41.115670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f6c50-9ccc-462d-89bb-c047b8779e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:41.116274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f6c50-9ccc-462d-89bb-c047b8779e3c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:41.116555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f6c50-9ccc-462d-89bb-c047b8779e3c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:41.116833 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48f6c50-9ccc-462d-89bb-c047b8779e3c None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:41.117526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f6c50-9ccc-462d-89bb-c047b8779e3c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:41.117830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f6c50-9ccc-462d-89bb-c047b8779e3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:41.119214 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48f6c50-9ccc-462d-89bb-c047b8779e3c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:41.127821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 40/82] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:58:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:41.128610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f6c50-9ccc-462d-89bb-c047b8779e3c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:41.170633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f3d830-d261-4b72-8199-165e51470eba None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:41.203735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03e4f0-99a0-442a-b95c-99525d9923d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:41.204424 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03e4f0-99a0-442a-b95c-99525d9923d4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:41.204712 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03e4f0-99a0-442a-b95c-99525d9923d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:41.204987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03e4f0-99a0-442a-b95c-99525d9923d4 None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:41.206137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03e4f0-99a0-442a-b95c-99525d9923d4 None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:41.207584 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03e4f0-99a0-442a-b95c-99525d9923d4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:41.214771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 43/83] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:41 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:41.214771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03e4f0-99a0-442a-b95c-99525d9923d4 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:41.241216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48f6c50-9ccc-462d-89bb-c047b8779e3c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:41.261045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:41.275530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d2882-2645-48f8-a42d-c877b01969f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:41.276715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d2882-2645-48f8-a42d-c877b01969f3 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:41.277000 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d2882-2645-48f8-a42d-c877b01969f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:41.277312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2d2882-2645-48f8-a42d-c877b01969f3 None admin] PATH_INFO: `/v3/users/fd866208c70e43639bb105515ea2921e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:41.278495 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d2882-2645-48f8-a42d-c877b01969f3 None admin] RBAC: Authorizing `identity:get_user(user_id=fd866208c70e43639bb105515ea2921e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:41.279945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2d2882-2645-48f8-a42d-c877b01969f3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:41.281564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 41/84] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:41 2022] GET /identity/v3/users/fd866208c70e43639bb105515ea2921e => generated 275 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:41.282327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d2882-2645-48f8-a42d-c877b01969f3 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:41.303859 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:41.314091 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03e4f0-99a0-442a-b95c-99525d9923d4 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:41.347754 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f23df7-8f5b-4eb6-b307-e059ba4ac8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:41.348359 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f23df7-8f5b-4eb6-b307-e059ba4ac8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:41.348648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f23df7-8f5b-4eb6-b307-e059ba4ac8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:41.348923 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f23df7-8f5b-4eb6-b307-e059ba4ac8f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:41.383574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2d2882-2645-48f8-a42d-c877b01969f3 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:41.415988 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f23df7-8f5b-4eb6-b307-e059ba4ac8f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:41.417576 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f23df7-8f5b-4eb6-b307-e059ba4ac8f2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:41.418492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc0c27-3a56-497a-9c36-214df7e666f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:41.419116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc0c27-3a56-497a-9c36-214df7e666f5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:41.419425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc0c27-3a56-497a-9c36-214df7e666f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:41.419695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc0c27-3a56-497a-9c36-214df7e666f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:41.420861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc0c27-3a56-497a-9c36-214df7e666f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:41.422256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfc0c27-3a56-497a-9c36-214df7e666f5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:41.423826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 42/85] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:58:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:41.424587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc0c27-3a56-497a-9c36-214df7e666f5 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:41.480307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:41.510116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:41.520890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 44/86] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:58:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:58:41.521838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f23df7-8f5b-4eb6-b307-e059ba4ac8f2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:41.524638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc0c27-3a56-497a-9c36-214df7e666f5 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:41.560259 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea9c7d-18d1-422d-b192-612cbe982d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:41.560885 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea9c7d-18d1-422d-b192-612cbe982d60 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:41.561163 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea9c7d-18d1-422d-b192-612cbe982d60 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:41.561466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ea9c7d-18d1-422d-b192-612cbe982d60 None admin] PATH_INFO: `/v3/users/fd866208c70e43639bb105515ea2921e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:41.562591 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea9c7d-18d1-422d-b192-612cbe982d60 None admin] RBAC: Authorizing `identity:get_user(user_id=fd866208c70e43639bb105515ea2921e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:41.564028 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ea9c7d-18d1-422d-b192-612cbe982d60 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:41.565741 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 43/87] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:41 2022] GET /identity/v3/users/fd866208c70e43639bb105515ea2921e => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:41.566497 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea9c7d-18d1-422d-b192-612cbe982d60 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:41.626680 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f23df7-8f5b-4eb6-b307-e059ba4ac8f2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:41.659973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802d1f4-2b96-4496-a644-623fddf47924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:41.660578 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802d1f4-2b96-4496-a644-623fddf47924 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:41.660860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802d1f4-2b96-4496-a644-623fddf47924 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:41.661142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802d1f4-2b96-4496-a644-623fddf47924 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:41.661892 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802d1f4-2b96-4496-a644-623fddf47924 None admin] RBAC: Adding query filter params (role.id=b80a0d1a8d164b53bef5726b92e68676, scope.project.id=7d21b4b8971a47ff8985309015eb53d4, user.id=fd866208c70e43639bb105515ea2921e) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:41.662201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802d1f4-2b96-4496-a644-623fddf47924 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:41.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802d1f4-2b96-4496-a644-623fddf47924 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:41.663775 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ea9c7d-18d1-422d-b192-612cbe982d60 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:41.671814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 45/88] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:58:41 2022] GET /identity/v3/role_assignments?user.id=fd866208c70e43639bb105515ea2921e&scope.project.id=7d21b4b8971a47ff8985309015eb53d4&role.id=b80a0d1a8d164b53bef5726b92e68676 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:41.672641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802d1f4-2b96-4496-a644-623fddf47924 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:41.688029 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:41.703926 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4046e6-340a-446f-8ce4-390d4b19ed70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:41.704719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4046e6-340a-446f-8ce4-390d4b19ed70 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:41.705006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4046e6-340a-446f-8ce4-390d4b19ed70 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:41.705685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4046e6-340a-446f-8ce4-390d4b19ed70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:41.737940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:41.785952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4046e6-340a-446f-8ce4-390d4b19ed70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:41.787459 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4046e6-340a-446f-8ce4-390d4b19ed70 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:41.788341 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802d1f4-2b96-4496-a644-623fddf47924 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:41.829351 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae220b78-1c2a-4859-aaed-cef88c62d99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:41.829973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae220b78-1c2a-4859-aaed-cef88c62d99c None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:41.830257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae220b78-1c2a-4859-aaed-cef88c62d99c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:41.830534 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae220b78-1c2a-4859-aaed-cef88c62d99c None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:41.831240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae220b78-1c2a-4859-aaed-cef88c62d99c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:41.832701 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae220b78-1c2a-4859-aaed-cef88c62d99c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:41.865226 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 46/89] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 18:58:41 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:58:41.865226 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae220b78-1c2a-4859-aaed-cef88c62d99c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:41.916271 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 44/90] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:58:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:58:41.917656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4046e6-340a-446f-8ce4-390d4b19ed70 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:41.946392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:42.006111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae220b78-1c2a-4859-aaed-cef88c62d99c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:42.021458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:42.054792 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec10f1-8641-43fc-bc5c-240967ef5d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:42.055103 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec10f1-8641-43fc-bc5c-240967ef5d37 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:42.055501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec10f1-8641-43fc-bc5c-240967ef5d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:42.057346 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec10f1-8641-43fc-bc5c-240967ef5d37 None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:42.058510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4046e6-340a-446f-8ce4-390d4b19ed70 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:42.058758 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec10f1-8641-43fc-bc5c-240967ef5d37 None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:42.060633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec10f1-8641-43fc-bc5c-240967ef5d37 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:42.063793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 47/91] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:41 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:42.063793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec10f1-8641-43fc-bc5c-240967ef5d37 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:42.100079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96393137-7c5e-43f4-9a12-92f41f1424b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:42.101044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96393137-7c5e-43f4-9a12-92f41f1424b6 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:42.101422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96393137-7c5e-43f4-9a12-92f41f1424b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:42.101774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96393137-7c5e-43f4-9a12-92f41f1424b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:42.190222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96393137-7c5e-43f4-9a12-92f41f1424b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:42.192215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96393137-7c5e-43f4-9a12-92f41f1424b6 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:42.199036 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec10f1-8641-43fc-bc5c-240967ef5d37 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:42.221156 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:42.254427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfbcf7-326f-4fa0-8117-d898d4e6e042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:42.255498 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfbcf7-326f-4fa0-8117-d898d4e6e042 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:42.255882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfbcf7-326f-4fa0-8117-d898d4e6e042 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:42.256328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6bfbcf7-326f-4fa0-8117-d898d4e6e042 None admin] PATH_INFO: `/v3/projects/ae6e53e6c2af4bfcab38f81ce10a6f58` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:42.257652 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfbcf7-326f-4fa0-8117-d898d4e6e042 None admin] RBAC: Authorizing `identity:get_project(project_id=ae6e53e6c2af4bfcab38f81ce10a6f58)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:42.259190 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6bfbcf7-326f-4fa0-8117-d898d4e6e042 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:42.261265 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 48/92] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:42 2022] GET /identity/v3/projects/ae6e53e6c2af4bfcab38f81ce10a6f58 => generated 312 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:42.262727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfbcf7-326f-4fa0-8117-d898d4e6e042 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:42.315317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 45/93] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:58:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:58:42.316320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96393137-7c5e-43f4-9a12-92f41f1424b6 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:42.329921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:42.404931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6bfbcf7-326f-4fa0-8117-d898d4e6e042 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:42.446171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b12883-fd38-4c5c-883f-13b1ff6c8f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:42.446814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b12883-fd38-4c5c-883f-13b1ff6c8f47 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:42.447076 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b12883-fd38-4c5c-883f-13b1ff6c8f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:42.447356 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b12883-fd38-4c5c-883f-13b1ff6c8f47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:42.448171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b12883-fd38-4c5c-883f-13b1ff6c8f47 None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=7d21b4b8971a47ff8985309015eb53d4, user.id=2d817dbb81e84447ac0c66c6621a6e8f) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:42.448488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b12883-fd38-4c5c-883f-13b1ff6c8f47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:42.450154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b12883-fd38-4c5c-883f-13b1ff6c8f47 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:42.455649 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 49/94] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:58:42 2022] GET /identity/v3/role_assignments?user.id=2d817dbb81e84447ac0c66c6621a6e8f&scope.project.id=7d21b4b8971a47ff8985309015eb53d4&role.id=3580c894dd974d35bc95848b8ba22598 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:42.456464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b12883-fd38-4c5c-883f-13b1ff6c8f47 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:42.467666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96393137-7c5e-43f4-9a12-92f41f1424b6 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:42.495080 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:42.509212 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d267884-388b-490d-b92a-7d63455740c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:42.510169 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d267884-388b-490d-b92a-7d63455740c9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:42.510709 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d267884-388b-490d-b92a-7d63455740c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:42.511177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d267884-388b-490d-b92a-7d63455740c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:42.512094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d267884-388b-490d-b92a-7d63455740c9 None admin] RBAC: Adding query filter params (role.id=b80a0d1a8d164b53bef5726b92e68676, scope.project.id=ae6e53e6c2af4bfcab38f81ce10a6f58, user.id=fd866208c70e43639bb105515ea2921e) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:42.512600 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d267884-388b-490d-b92a-7d63455740c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:42.514352 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d267884-388b-490d-b92a-7d63455740c9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:42.521315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 46/95] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:58:42 2022] GET /identity/v3/role_assignments?user.id=fd866208c70e43639bb105515ea2921e&scope.project.id=ae6e53e6c2af4bfcab38f81ce10a6f58&role.id=b80a0d1a8d164b53bef5726b92e68676 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:42.574864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b12883-fd38-4c5c-883f-13b1ff6c8f47 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:42.586860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:42.614708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d669f996-fe86-41f8-994c-d663c4fd39f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:42.615366 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669f996-fe86-41f8-994c-d663c4fd39f2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:42.615651 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669f996-fe86-41f8-994c-d663c4fd39f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:42.615931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d669f996-fe86-41f8-994c-d663c4fd39f2 None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:42.617385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669f996-fe86-41f8-994c-d663c4fd39f2 None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:42.618703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d669f996-fe86-41f8-994c-d663c4fd39f2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:42.620857 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 50/96] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:42 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:42.628294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d267884-388b-490d-b92a-7d63455740c9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:42.759816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d267884-388b-490d-b92a-7d63455740c9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:42.803464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a04336-bb81-4089-ac82-a6c80f2d6b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:42.804084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a04336-bb81-4089-ac82-a6c80f2d6b2e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:42.804365 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a04336-bb81-4089-ac82-a6c80f2d6b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:42.804641 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a04336-bb81-4089-ac82-a6c80f2d6b2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:42.805417 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a04336-bb81-4089-ac82-a6c80f2d6b2e None admin] RBAC: Adding query filter params (role.id=470b660efbe04083a803e79d8b647160, scope.project.id=7d21b4b8971a47ff8985309015eb53d4, user.id=fd866208c70e43639bb105515ea2921e) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:42.805731 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a04336-bb81-4089-ac82-a6c80f2d6b2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:42.807187 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a04336-bb81-4089-ac82-a6c80f2d6b2e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:42.815169 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 47/97] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:58:42 2022] GET /identity/v3/role_assignments?user.id=fd866208c70e43639bb105515ea2921e&scope.project.id=7d21b4b8971a47ff8985309015eb53d4&role.id=470b660efbe04083a803e79d8b647160 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:44.294533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c8528-3aa9-43b6-bbc3-f0aaec23fac6 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:44.295034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c8528-3aa9-43b6-bbc3-f0aaec23fac6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:44.295106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c8528-3aa9-43b6-bbc3-f0aaec23fac6 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:44.296700 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 51/98] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:44.302135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37d428-f5ac-4985-b3fb-ce0c7ebc81fb None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:44.302523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37d428-f5ac-4985-b3fb-ce0c7ebc81fb None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:44.302684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37d428-f5ac-4985-b3fb-ce0c7ebc81fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:44.324316 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-fe37d428-f5ac-4985-b3fb-ce0c7ebc81fb None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:44.325636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-fe37d428-f5ac-4985-b3fb-ce0c7ebc81fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:44.382190 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:44.408642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 48/99] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:44.936006 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c8528-3aa9-43b6-bbc3-f0aaec23fac6 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:45.015325 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776ed47-5ea7-40c7-b5fd-874bbc47dea0 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:45.015771 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776ed47-5ea7-40c7-b5fd-874bbc47dea0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:45.015955 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a776ed47-5ea7-40c7-b5fd-874bbc47dea0 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:45.018026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 49/100] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 18:58:45.023901 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebb06b-6ca9-4f75-a934-7da7af4f044e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:45.024189 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebb06b-6ca9-4f75-a934-7da7af4f044e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:45.024469 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ebb06b-6ca9-4f75-a934-7da7af4f044e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:45.059750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-04ebb06b-6ca9-4f75-a934-7da7af4f044e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:45.060991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-04ebb06b-6ca9-4f75-a934-7da7af4f044e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:45.121660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c8528-3aa9-43b6-bbc3-f0aaec23fac6 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:45.181640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 50/101] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:58:45.182748 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1bce5-9921-48af-b672-49d65d9190a6 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:45.183037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1bce5-9921-48af-b672-49d65d9190a6 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:45.183315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1bce5-9921-48af-b672-49d65d9190a6 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:45.189741 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd0f7f-3b22-412e-b0a9-a26d2cf4a816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:45.190330 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd0f7f-3b22-412e-b0a9-a26d2cf4a816 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:45.190615 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd0f7f-3b22-412e-b0a9-a26d2cf4a816 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:45.190895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd0f7f-3b22-412e-b0a9-a26d2cf4a816 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:45.191712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 51/102] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:45.202638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb11f1e-ccb4-4b53-a525-a89142574d53 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:45.202928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb11f1e-ccb4-4b53-a525-a89142574d53 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:45.203277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb11f1e-ccb4-4b53-a525-a89142574d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:45.228887 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd0f7f-3b22-412e-b0a9-a26d2cf4a816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:45.242433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-23cd0f7f-3b22-412e-b0a9-a26d2cf4a816 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:58:45.242433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:45.242433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:45.242433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:45.242433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:45.242433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:45.242433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:45.242433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:45.242433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:45.242433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:45.242433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:45.242433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:45.242433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:45.242433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:45.242433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:45.242433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:45.243338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:45.243338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:45.243338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:45.243338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:45.243338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:45.243338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:45.243338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:45.243338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:45.243338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:45.243338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:45.243338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:45.243338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:45.243338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:45.243338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:45.243338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:45.243338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:45.244074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:45.244074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:45.244074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:45.244074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:45.244074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:45.244074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:45.244074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:45.244074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:45.244074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:45.244074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:45.244074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:58:45.244074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:58:45.244667 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9cb11f1e-ccb4-4b53-a525-a89142574d53 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:45.246158 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9cb11f1e-ccb4-4b53-a525-a89142574d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:45.247096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 52/103] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:45.252009 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd0f7f-3b22-412e-b0a9-a26d2cf4a816 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:45.347468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:45.351222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 52/104] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:45.353252 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde48ec1-c3c3-47a1-b8bf-00785ceee5ce None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:45.353825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde48ec1-c3c3-47a1-b8bf-00785ceee5ce None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:45.354329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde48ec1-c3c3-47a1-b8bf-00785ceee5ce None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:45.355874 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 53/105] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:45.363264 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822287b6-8658-47c2-8b04-791ebd208b0a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:45.363835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822287b6-8658-47c2-8b04-791ebd208b0a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:45.364305 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822287b6-8658-47c2-8b04-791ebd208b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:45.406206 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-822287b6-8658-47c2-8b04-791ebd208b0a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:45.407408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-822287b6-8658-47c2-8b04-791ebd208b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:45.449591 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd0f7f-3b22-412e-b0a9-a26d2cf4a816 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:45.478424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:45.496099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6597c-64ca-4394-99a1-400e5b1dbb92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:45.497752 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6597c-64ca-4394-99a1-400e5b1dbb92 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:45.498257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6597c-64ca-4394-99a1-400e5b1dbb92 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:45.498730 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e6597c-64ca-4394-99a1-400e5b1dbb92 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:45.499592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6597c-64ca-4394-99a1-400e5b1dbb92 None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:45.500580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6597c-64ca-4394-99a1-400e5b1dbb92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:45.502802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e6597c-64ca-4394-99a1-400e5b1dbb92 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:45.514854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 54/106] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:58:45.515739 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 53/107] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:58:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:45.521597 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1cb23-dcf8-48b2-a3fb-ce35e7c1b487 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:45.521890 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1cb23-dcf8-48b2-a3fb-ce35e7c1b487 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:45.522235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a1cb23-dcf8-48b2-a3fb-ce35e7c1b487 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:45.523538 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 55/108] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:45 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:45.526563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6597c-64ca-4394-99a1-400e5b1dbb92 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:45.528286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1cb23-dcf8-48b2-a3fb-ce35e7c1b487 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:45.708267 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e6597c-64ca-4394-99a1-400e5b1dbb92 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:45.717874 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:45.727313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a1cb23-dcf8-48b2-a3fb-ce35e7c1b487 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:45.750281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a4e1e4-c755-439d-b190-cd0876830958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:45.750920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4e1e4-c755-439d-b190-cd0876830958 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:45.751203 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4e1e4-c755-439d-b190-cd0876830958 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:45.751479 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a4e1e4-c755-439d-b190-cd0876830958 None admin] PATH_INFO: `/v3/users/fd866208c70e43639bb105515ea2921e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:45.752597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a4e1e4-c755-439d-b190-cd0876830958 None admin] RBAC: Authorizing `identity:get_user(user_id=fd866208c70e43639bb105515ea2921e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:45.754512 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a4e1e4-c755-439d-b190-cd0876830958 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:45.756082 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 54/109] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:45 2022] GET /identity/v3/users/fd866208c70e43639bb105515ea2921e => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:45.757164 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedb8be-429f-47cd-93b9-50caa8bad4bd None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:45.757465 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedb8be-429f-47cd-93b9-50caa8bad4bd None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:45.757735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afedb8be-429f-47cd-93b9-50caa8bad4bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:45.771869 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a74fd28-0a96-47e6-af85-667ca6c49eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:45.772751 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74fd28-0a96-47e6-af85-667ca6c49eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:45.773256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74fd28-0a96-47e6-af85-667ca6c49eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:45.773701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a74fd28-0a96-47e6-af85-667ca6c49eb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:45.775078 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-afedb8be-429f-47cd-93b9-50caa8bad4bd None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:45.776283 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-afedb8be-429f-47cd-93b9-50caa8bad4bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:45.784828 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a74fd28-0a96-47e6-af85-667ca6c49eb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:45.789888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-0a74fd28-0a96-47e6-af85-667ca6c49eb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:58:45.789888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:45.789888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:45.789888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:45.789888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:45.789888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:45.789888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:45.789888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:45.789888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:45.789888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:45.789888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:45.789888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:45.789888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:45.789888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:45.789888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:45.789888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:45.790739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:45.790739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:45.790739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:45.790739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:45.790739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:45.790739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:45.790739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:45.790739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:45.790739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:45.790739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:45.790739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:45.790739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:45.790739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:45.790739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:45.790739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:45.790739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:45.793929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:45.793929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:45.793929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:45.793929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:45.793929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:45.793929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:45.793929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:45.793929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:45.793929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:45.793929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:45.793929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:58:45.793929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:58:45.793929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 56/110] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:58:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 18:58:45.793929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02c2d0-38f4-4460-a33c-7fa5f7284be3 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:45.794602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02c2d0-38f4-4460-a33c-7fa5f7284be3 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:45.794602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de02c2d0-38f4-4460-a33c-7fa5f7284be3 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:45.794722 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 57/111] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:45.795345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02c2d0-38f4-4460-a33c-7fa5f7284be3 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:45.832313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:45.855488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 55/112] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:45.857319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedb8be-429f-47cd-93b9-50caa8bad4bd None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:45.912805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de02c2d0-38f4-4460-a33c-7fa5f7284be3 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:45.948090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278aa40-8e3f-4204-ae1e-0507d3a8c22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:45.948725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278aa40-8e3f-4204-ae1e-0507d3a8c22f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:45.949003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278aa40-8e3f-4204-ae1e-0507d3a8c22f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:45.949306 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3278aa40-8e3f-4204-ae1e-0507d3a8c22f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:45.954353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:45.959094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afedb8be-429f-47cd-93b9-50caa8bad4bd None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:45.960361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3278aa40-8e3f-4204-ae1e-0507d3a8c22f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:45.965181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-3278aa40-8e3f-4204-ae1e-0507d3a8c22f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:58:45.965181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:45.965181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:45.965181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:45.965181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:45.965181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:45.965181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:45.965181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:45.965181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:45.965181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:45.965181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:45.965181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:45.965181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:45.965181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:45.965181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:45.965181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:45.966039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:45.966039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:45.966039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:45.966039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:45.966039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:45.966039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:45.966039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:45.966039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:45.966039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:45.966039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:45.966039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:45.966039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:45.966039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:45.966039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:45.966039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:45.966039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:45.966797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:45.966797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:45.966797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:45.966797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:45.966797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:45.966797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:45.966797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:45.966797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:45.966797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:45.966797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:45.966797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:58:45.966797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:58:45.966797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 58/113] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:45.967439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278aa40-8e3f-4204-ae1e-0507d3a8c22f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:46.002405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb88c64-7fbc-4d50-8e96-2cea1b68c0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:46.003056 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb88c64-7fbc-4d50-8e96-2cea1b68c0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:46.003321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb88c64-7fbc-4d50-8e96-2cea1b68c0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:46.003596 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb88c64-7fbc-4d50-8e96-2cea1b68c0a9 None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:46.005350 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb88c64-7fbc-4d50-8e96-2cea1b68c0a9 None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:46.006782 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb88c64-7fbc-4d50-8e96-2cea1b68c0a9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:46.008630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 56/114] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:45 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:46.010250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb88c64-7fbc-4d50-8e96-2cea1b68c0a9 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:46.067381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:46.077406 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3278aa40-8e3f-4204-ae1e-0507d3a8c22f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:46.113428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bbb918-a31b-42b7-8a32-1c4516870b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:46.114110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbb918-a31b-42b7-8a32-1c4516870b01 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:46.114397 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbb918-a31b-42b7-8a32-1c4516870b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:46.114671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bbb918-a31b-42b7-8a32-1c4516870b01 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:46.115785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbb918-a31b-42b7-8a32-1c4516870b01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:46.115881 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb88c64-7fbc-4d50-8e96-2cea1b68c0a9 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:46.117258 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bbb918-a31b-42b7-8a32-1c4516870b01 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:46.119123 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 59/115] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:58:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:46.120270 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89f403-11dc-43d8-82e6-aa3d11cf54a8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:46.120552 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89f403-11dc-43d8-82e6-aa3d11cf54a8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:46.120824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a89f403-11dc-43d8-82e6-aa3d11cf54a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:46.139122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4a89f403-11dc-43d8-82e6-aa3d11cf54a8 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:46.140369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4a89f403-11dc-43d8-82e6-aa3d11cf54a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:46.150919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c134267-3f52-451f-bc58-645f17fa5de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:46.152050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c134267-3f52-451f-bc58-645f17fa5de7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:46.152325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c134267-3f52-451f-bc58-645f17fa5de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:46.152592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c134267-3f52-451f-bc58-645f17fa5de7 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:46.153273 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c134267-3f52-451f-bc58-645f17fa5de7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:46.153576 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c134267-3f52-451f-bc58-645f17fa5de7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:46.154927 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c134267-3f52-451f-bc58-645f17fa5de7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:46.167805 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 57/116] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:46.167805 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c134267-3f52-451f-bc58-645f17fa5de7 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:46.191606 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:46.206570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 60/117] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:46.207545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89f403-11dc-43d8-82e6-aa3d11cf54a8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:46.269735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c134267-3f52-451f-bc58-645f17fa5de7 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:46.307038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a154d-a6e3-4909-a966-dd0d203de45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:46.307648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a154d-a6e3-4909-a966-dd0d203de45e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:46.307931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a154d-a6e3-4909-a966-dd0d203de45e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:46.308203 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a154d-a6e3-4909-a966-dd0d203de45e None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:46.308861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a154d-a6e3-4909-a966-dd0d203de45e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:46.309165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a154d-a6e3-4909-a966-dd0d203de45e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:46.310568 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a154d-a6e3-4909-a966-dd0d203de45e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:46.311273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:46.319545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 58/118] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:58:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:46.320338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a154d-a6e3-4909-a966-dd0d203de45e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:46.326537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a89f403-11dc-43d8-82e6-aa3d11cf54a8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:46.362286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6d147-2290-416a-b3a5-4efc3ddbd0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:46.362959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6d147-2290-416a-b3a5-4efc3ddbd0cb None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:46.363240 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6d147-2290-416a-b3a5-4efc3ddbd0cb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:46.363526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba6d147-2290-416a-b3a5-4efc3ddbd0cb None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4/users/fd866208c70e43639bb105515ea2921e/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:46.365457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6d147-2290-416a-b3a5-4efc3ddbd0cb None admin] RBAC: Authorizing `identity:create_grant(project_id=7d21b4b8971a47ff8985309015eb53d4, user_id=fd866208c70e43639bb105515ea2921e, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:46.366902 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba6d147-2290-416a-b3a5-4efc3ddbd0cb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:46.381221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 61/119] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 18:58:46 2022] PUT /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4/users/fd866208c70e43639bb105515ea2921e/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 18:58:46.390258 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6d147-2290-416a-b3a5-4efc3ddbd0cb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:46.425832 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:46.437952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a154d-a6e3-4909-a966-dd0d203de45e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:46.478480 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2a9b0-490b-4761-84d6-bf9fcf011fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:46.479194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2a9b0-490b-4761-84d6-bf9fcf011fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:46.480152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2a9b0-490b-4761-84d6-bf9fcf011fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:46.480463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2a9b0-490b-4761-84d6-bf9fcf011fa4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:46.492644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2a9b0-490b-4761-84d6-bf9fcf011fa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:46.498573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-f4f2a9b0-490b-4761-84d6-bf9fcf011fa4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:58:46.498573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:46.498573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:46.498573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:46.498573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:46.498573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:46.498573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:46.498573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:46.498573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:46.498573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:46.498573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:46.498573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:46.498573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:46.498573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:46.498573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:46.498573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:46.499524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:46.499524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:46.499524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:46.499524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:46.499524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:46.499524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:46.499524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:46.499524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:46.499524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:46.499524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:46.499524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:46.499524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:46.499524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:46.499524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:46.499524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:46.499524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:46.505293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:46.505293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:46.505293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:46.505293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:46.505293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:46.505293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:46.505293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:46.505293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:46.505293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:46.505293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:46.505293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:58:46.505293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:58:46.505293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 59/120] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 18:58:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:46.505293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2a9b0-490b-4761-84d6-bf9fcf011fa4 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:46.542517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba6d147-2290-416a-b3a5-4efc3ddbd0cb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:46.578322 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28584b1-7492-4752-8ac6-20e7720827e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:46.578937 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28584b1-7492-4752-8ac6-20e7720827e4 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:46.579220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28584b1-7492-4752-8ac6-20e7720827e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:46.579494 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28584b1-7492-4752-8ac6-20e7720827e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:46.580143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28584b1-7492-4752-8ac6-20e7720827e4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:46.581638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28584b1-7492-4752-8ac6-20e7720827e4 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:46.611106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 62/121] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 18:58:46 2022] POST /identity/v3/projects => generated 302 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:58:46.611752 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28584b1-7492-4752-8ac6-20e7720827e4 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:46.614373 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:46.626577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2a9b0-490b-4761-84d6-bf9fcf011fa4 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:46.667520 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb4b25-5079-4eef-827d-227ff57f8211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:46.668414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb4b25-5079-4eef-827d-227ff57f8211 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:46.668875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb4b25-5079-4eef-827d-227ff57f8211 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:46.669328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb4b25-5079-4eef-827d-227ff57f8211 None admin] PATH_INFO: `/v3/users/2d817dbb81e84447ac0c66c6621a6e8f` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:46.670925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb4b25-5079-4eef-827d-227ff57f8211 None admin] RBAC: Authorizing `identity:get_user(user_id=2d817dbb81e84447ac0c66c6621a6e8f)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:46.672423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb4b25-5079-4eef-827d-227ff57f8211 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:46.678899 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 60/122] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:46 2022] GET /identity/v3/users/2d817dbb81e84447ac0c66c6621a6e8f => generated 247 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:46.678899 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb4b25-5079-4eef-827d-227ff57f8211 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:46.701157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:46.770687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28584b1-7492-4752-8ac6-20e7720827e4 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:46.801011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb4b25-5079-4eef-827d-227ff57f8211 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:46.809262 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65357ed-cb3b-4814-99f5-61444145825d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:46.809998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65357ed-cb3b-4814-99f5-61444145825d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:46.811310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65357ed-cb3b-4814-99f5-61444145825d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:46.811310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65357ed-cb3b-4814-99f5-61444145825d None admin] PATH_INFO: `/v3/users/0b6cfad5da8e49df8e594be53d842ea5` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:46.828770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65357ed-cb3b-4814-99f5-61444145825d None admin] RBAC: Authorizing `identity:get_user(user_id=0b6cfad5da8e49df8e594be53d842ea5)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:46.831387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65357ed-cb3b-4814-99f5-61444145825d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:46.834139 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 63/123] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:46 2022] GET /identity/v3/users/0b6cfad5da8e49df8e594be53d842ea5 => generated 289 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:46.838268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65357ed-cb3b-4814-99f5-61444145825d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:46.854930 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0216b7-9056-4dfc-8cd2-fe6a7851a57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:46.854930 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0216b7-9056-4dfc-8cd2-fe6a7851a57b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:46.855225 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0216b7-9056-4dfc-8cd2-fe6a7851a57b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:46.855492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0216b7-9056-4dfc-8cd2-fe6a7851a57b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:46.874120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0216b7-9056-4dfc-8cd2-fe6a7851a57b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:46.881421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-1b0216b7-9056-4dfc-8cd2-fe6a7851a57b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:58:46.881421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:46.881421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:46.881421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:46.881421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:46.881421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:46.881421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:46.881421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:46.881421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:46.881421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:46.881421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:46.881421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:46.881421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:46.881421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:46.881421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:46.881421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:46.882420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:46.882420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:46.882420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:46.882420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:46.882420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:46.882420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:46.882420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:46.882420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:46.882420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:46.882420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:46.882420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:46.882420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:46.882420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:46.882420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:46.882420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:46.882420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:46.883315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:46.883315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:46.883315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:46.883315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:46.883315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:46.883315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:46.883315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:46.883315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:46.883315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:46.883315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:46.883315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:58:46.883315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:58:46.885289 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 61/124] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:46.886213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0216b7-9056-4dfc-8cd2-fe6a7851a57b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:46.908046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:46.965093 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65357ed-cb3b-4814-99f5-61444145825d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:46.982490 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:47.011507 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0216b7-9056-4dfc-8cd2-fe6a7851a57b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:47.016477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df136f5-d7c6-4f6f-87ae-1dcf9e8f18b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:47.017214 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df136f5-d7c6-4f6f-87ae-1dcf9e8f18b5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:47.017611 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df136f5-d7c6-4f6f-87ae-1dcf9e8f18b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:47.017981 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df136f5-d7c6-4f6f-87ae-1dcf9e8f18b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:47.019264 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df136f5-d7c6-4f6f-87ae-1dcf9e8f18b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:47.019685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df136f5-d7c6-4f6f-87ae-1dcf9e8f18b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:47.021656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df136f5-d7c6-4f6f-87ae-1dcf9e8f18b5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:47.034119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 64/125] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:58:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:47.035185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df136f5-d7c6-4f6f-87ae-1dcf9e8f18b5 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:47.052741 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4135d5c-823d-4995-933f-0e798cf5f6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:47.053378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4135d5c-823d-4995-933f-0e798cf5f6db None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:47.053668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4135d5c-823d-4995-933f-0e798cf5f6db None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:47.053945 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4135d5c-823d-4995-933f-0e798cf5f6db None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:47.055075 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4135d5c-823d-4995-933f-0e798cf5f6db None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:47.056538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4135d5c-823d-4995-933f-0e798cf5f6db None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:47.058752 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 62/126] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:46 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:47.059511 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4135d5c-823d-4995-933f-0e798cf5f6db None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:47.158868 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df136f5-d7c6-4f6f-87ae-1dcf9e8f18b5 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:47.181051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4135d5c-823d-4995-933f-0e798cf5f6db None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:47.184772 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:47.202539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838f543-7c49-40a6-b464-f85083691db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:47.203565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838f543-7c49-40a6-b464-f85083691db9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:47.204010 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838f543-7c49-40a6-b464-f85083691db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:47.204664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7838f543-7c49-40a6-b464-f85083691db9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:47.229424 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a239961-986a-4fb7-a99d-ac19953479cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:47.230795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a239961-986a-4fb7-a99d-ac19953479cc None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:47.231187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a239961-986a-4fb7-a99d-ac19953479cc None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:47.231557 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a239961-986a-4fb7-a99d-ac19953479cc None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:47.232323 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a239961-986a-4fb7-a99d-ac19953479cc None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:47.232734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a239961-986a-4fb7-a99d-ac19953479cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:47.234522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a239961-986a-4fb7-a99d-ac19953479cc None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:47.247884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 63/127] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:58:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:47.247884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a239961-986a-4fb7-a99d-ac19953479cc None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:47.259533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:47.301742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838f543-7c49-40a6-b464-f85083691db9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:47.303861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7838f543-7c49-40a6-b464-f85083691db9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:47.373112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a239961-986a-4fb7-a99d-ac19953479cc None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:47.411174 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 65/128] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:58:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:58:47.412816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838f543-7c49-40a6-b464-f85083691db9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:47.427239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4318a0-3d27-4e59-a91b-57d53daadc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:47.427886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4318a0-3d27-4e59-a91b-57d53daadc74 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:47.428171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4318a0-3d27-4e59-a91b-57d53daadc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:47.428446 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4318a0-3d27-4e59-a91b-57d53daadc74 None admin] PATH_INFO: `/v3/users/fd866208c70e43639bb105515ea2921e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:47.429918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4318a0-3d27-4e59-a91b-57d53daadc74 None admin] RBAC: Authorizing `identity:get_user(user_id=fd866208c70e43639bb105515ea2921e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:47.431647 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4318a0-3d27-4e59-a91b-57d53daadc74 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:47.434049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 64/129] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:47 2022] GET /identity/v3/users/fd866208c70e43639bb105515ea2921e => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:47.435058 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4318a0-3d27-4e59-a91b-57d53daadc74 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:47.442695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:47.528911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:47.545858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7838f543-7c49-40a6-b464-f85083691db9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:47.561551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4318a0-3d27-4e59-a91b-57d53daadc74 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:47.584942 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd4771-bdf6-4bb2-bf22-9ecda8070bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:47.585636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd4771-bdf6-4bb2-bf22-9ecda8070bd8 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:47.585921 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd4771-bdf6-4bb2-bf22-9ecda8070bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:47.586199 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd4771-bdf6-4bb2-bf22-9ecda8070bd8 None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4/users/2d817dbb81e84447ac0c66c6621a6e8f/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:47.589471 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd4771-bdf6-4bb2-bf22-9ecda8070bd8 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d21b4b8971a47ff8985309015eb53d4, user_id=2d817dbb81e84447ac0c66c6621a6e8f, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:47.591583 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd4771-bdf6-4bb2-bf22-9ecda8070bd8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:47.604846 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1aa1b-d526-4671-b756-a674486f37b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:47.605592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 66/130] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 18:58:47 2022] PUT /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4/users/2d817dbb81e84447ac0c66c6621a6e8f/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 18:58:47.606156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1aa1b-d526-4671-b756-a674486f37b0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:47.606626 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1aa1b-d526-4671-b756-a674486f37b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:47.607224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1aa1b-d526-4671-b756-a674486f37b0 None admin] PATH_INFO: `/v3/users/fd866208c70e43639bb105515ea2921e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:47.608636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1aa1b-d526-4671-b756-a674486f37b0 None admin] RBAC: Authorizing `identity:get_user(user_id=fd866208c70e43639bb105515ea2921e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:47.610433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1aa1b-d526-4671-b756-a674486f37b0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:47.617228 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd4771-bdf6-4bb2-bf22-9ecda8070bd8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:47.617642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 65/131] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:47 2022] GET /identity/v3/users/fd866208c70e43639bb105515ea2921e => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:47.619052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1aa1b-d526-4671-b756-a674486f37b0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:47.796888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:47.797632 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1aa1b-d526-4671-b756-a674486f37b0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:47.819574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd4771-bdf6-4bb2-bf22-9ecda8070bd8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:47.841775 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192cb49-ba16-4244-858f-3de37a791e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:47.842420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192cb49-ba16-4244-858f-3de37a791e5c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:47.842705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192cb49-ba16-4244-858f-3de37a791e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:47.842984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a192cb49-ba16-4244-858f-3de37a791e5c None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:47.844129 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a192cb49-ba16-4244-858f-3de37a791e5c None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:47.845716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a192cb49-ba16-4244-858f-3de37a791e5c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:47.847626 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 66/132] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:47 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:47.848207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192cb49-ba16-4244-858f-3de37a791e5c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:47.862250 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35ef5c-4b7c-4f5e-9740-3a8e1bc10013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:47.863210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35ef5c-4b7c-4f5e-9740-3a8e1bc10013 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:47.863751 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35ef5c-4b7c-4f5e-9740-3a8e1bc10013 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:47.864228 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc35ef5c-4b7c-4f5e-9740-3a8e1bc10013 None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:47.865791 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:47.866463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35ef5c-4b7c-4f5e-9740-3a8e1bc10013 None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:47.868163 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc35ef5c-4b7c-4f5e-9740-3a8e1bc10013 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:47.870248 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 67/133] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:47 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:47.871287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35ef5c-4b7c-4f5e-9740-3a8e1bc10013 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:47.963206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a192cb49-ba16-4244-858f-3de37a791e5c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:47.977574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc35ef5c-4b7c-4f5e-9740-3a8e1bc10013 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:48.001003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d222b1-6f0f-4416-a344-d08f2931fbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:48.001748 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d222b1-6f0f-4416-a344-d08f2931fbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:48.001956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d222b1-6f0f-4416-a344-d08f2931fbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:48.002229 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d222b1-6f0f-4416-a344-d08f2931fbe9 None admin] PATH_INFO: `/v3/projects/ae6e53e6c2af4bfcab38f81ce10a6f58` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:48.003374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d222b1-6f0f-4416-a344-d08f2931fbe9 None admin] RBAC: Authorizing `identity:get_project(project_id=ae6e53e6c2af4bfcab38f81ce10a6f58)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:48.005091 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d222b1-6f0f-4416-a344-d08f2931fbe9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:48.010065 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 67/134] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:47 2022] GET /identity/v3/projects/ae6e53e6c2af4bfcab38f81ce10a6f58 => generated 312 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:48.010065 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d222b1-6f0f-4416-a344-d08f2931fbe9 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:48.021045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee48527-9a6c-47ae-9fdc-e103d5940c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:48.021974 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee48527-9a6c-47ae-9fdc-e103d5940c29 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:48.022399 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee48527-9a6c-47ae-9fdc-e103d5940c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:48.022816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee48527-9a6c-47ae-9fdc-e103d5940c29 None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4/users/fd866208c70e43639bb105515ea2921e/roles/470b660efbe04083a803e79d8b647160` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:48.025209 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee48527-9a6c-47ae-9fdc-e103d5940c29 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d21b4b8971a47ff8985309015eb53d4, user_id=fd866208c70e43639bb105515ea2921e, role_id=470b660efbe04083a803e79d8b647160)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:48.026914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee48527-9a6c-47ae-9fdc-e103d5940c29 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:48.050301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 68/135] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 18:58:47 2022] PUT /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4/users/fd866208c70e43639bb105515ea2921e/roles/470b660efbe04083a803e79d8b647160 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 18:58:48.051416 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee48527-9a6c-47ae-9fdc-e103d5940c29 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:48.075312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:48.106133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:48.129961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d222b1-6f0f-4416-a344-d08f2931fbe9 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:48.170240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f70547-6bfe-4f06-976b-946cdfad5b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:48.171324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f70547-6bfe-4f06-976b-946cdfad5b56 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:48.171855 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f70547-6bfe-4f06-976b-946cdfad5b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:48.172325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f70547-6bfe-4f06-976b-946cdfad5b56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:48.173300 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f70547-6bfe-4f06-976b-946cdfad5b56 None admin] RBAC: Adding query filter params (role.id=2d35524d275d4bc4b09ce775ddec4963, scope.project.id=7d21b4b8971a47ff8985309015eb53d4, user.id=0b6cfad5da8e49df8e594be53d842ea5) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:48.173902 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f70547-6bfe-4f06-976b-946cdfad5b56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:48.175565 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f70547-6bfe-4f06-976b-946cdfad5b56 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:48.183061 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 68/136] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:58:48 2022] GET /identity/v3/role_assignments?user.id=0b6cfad5da8e49df8e594be53d842ea5&scope.project.id=7d21b4b8971a47ff8985309015eb53d4&role.id=2d35524d275d4bc4b09ce775ddec4963 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:48.251802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee48527-9a6c-47ae-9fdc-e103d5940c29 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:48.302741 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f2c48d-25c0-4565-9e4e-73169fed78b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:48.303435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2c48d-25c0-4565-9e4e-73169fed78b6 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:48.303717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2c48d-25c0-4565-9e4e-73169fed78b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:48.303992 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f2c48d-25c0-4565-9e4e-73169fed78b6 None admin] PATH_INFO: `/v3/projects/ae6e53e6c2af4bfcab38f81ce10a6f58/users/fd866208c70e43639bb105515ea2921e/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:48.306047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2c48d-25c0-4565-9e4e-73169fed78b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=ae6e53e6c2af4bfcab38f81ce10a6f58, user_id=fd866208c70e43639bb105515ea2921e, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:48.308199 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f2c48d-25c0-4565-9e4e-73169fed78b6 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:48.328030 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 69/137] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 18:58:48 2022] PUT /identity/v3/projects/ae6e53e6c2af4bfcab38f81ce10a6f58/users/fd866208c70e43639bb105515ea2921e/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 18:58:48.608851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07bdcf-d827-42ae-ace4-e1faa3278992 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:48.609374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07bdcf-d827-42ae-ace4-e1faa3278992 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:48.609374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07bdcf-d827-42ae-ace4-e1faa3278992 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:48.610858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 69/138] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:48.616053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a1103-ab3d-44e5-8834-276790807804 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:48.616473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a1103-ab3d-44e5-8834-276790807804 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:48.616643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3a1103-ab3d-44e5-8834-276790807804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:48.642649 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8b3a1103-ab3d-44e5-8834-276790807804 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:48.643846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8b3a1103-ab3d-44e5-8834-276790807804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:48.737208 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:48.743597 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 70/139] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:58:48.865830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac352f9c-50d3-49a6-b6cf-e61987f8c077 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:48.866130 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac352f9c-50d3-49a6-b6cf-e61987f8c077 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:48.866306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac352f9c-50d3-49a6-b6cf-e61987f8c077 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:48.867958 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 70/140] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:48.877030 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111ae67-fc79-483a-8114-d6ffaa92f933 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:48.877355 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111ae67-fc79-483a-8114-d6ffaa92f933 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:48.877690 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2111ae67-fc79-483a-8114-d6ffaa92f933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:48.913269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2111ae67-fc79-483a-8114-d6ffaa92f933 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:48.914470 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2111ae67-fc79-483a-8114-d6ffaa92f933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:48.997750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 71/141] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:58:49.162202 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac352f9c-50d3-49a6-b6cf-e61987f8c077 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:49.274311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac352f9c-50d3-49a6-b6cf-e61987f8c077 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:49.311744 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7b6ca-9163-403f-84e2-2dae1e9baa3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:49.312380 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7b6ca-9163-403f-84e2-2dae1e9baa3e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:49.312663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7b6ca-9163-403f-84e2-2dae1e9baa3e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:49.312947 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc7b6ca-9163-403f-84e2-2dae1e9baa3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:49.325300 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc7b6ca-9163-403f-84e2-2dae1e9baa3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:49.330154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-5cc7b6ca-9163-403f-84e2-2dae1e9baa3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:58:49.330154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:49.330154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:49.330154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:49.330154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:49.330154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:49.330154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:49.330154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:49.330154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:49.330154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:49.330154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:49.330154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:49.330154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:49.330154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:49.330154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:49.330154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:49.330956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:49.330956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:49.330956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:49.330956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:49.330956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:49.330956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:49.330956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:49.330956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:49.330956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:49.330956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:49.330956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:49.330956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:49.330956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:49.330956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:49.330956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:49.330956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:49.331703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:49.331703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:49.331703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:49.331703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:49.331703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:49.331703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:49.331703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:49.331703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:49.331703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:49.331703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:49.331703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:58:49.331703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:58:49.331703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 71/142] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:49.337433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2111ae67-fc79-483a-8114-d6ffaa92f933 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:49.479530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2111ae67-fc79-483a-8114-d6ffaa92f933 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:49.517848 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:49.524809 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593fa12-9236-447f-8c1d-6de2d36b9ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:49.525449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593fa12-9236-447f-8c1d-6de2d36b9ab1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:49.525734 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593fa12-9236-447f-8c1d-6de2d36b9ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:49.526012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6593fa12-9236-447f-8c1d-6de2d36b9ab1 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:49.527047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593fa12-9236-447f-8c1d-6de2d36b9ab1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:49.527047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593fa12-9236-447f-8c1d-6de2d36b9ab1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:49.528559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6593fa12-9236-447f-8c1d-6de2d36b9ab1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:49.537293 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 72/143] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:58:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:49.543133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7b6ca-9163-403f-84e2-2dae1e9baa3e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:49.547085 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:49.576526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593fa12-9236-447f-8c1d-6de2d36b9ab1 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:49.673841 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc7b6ca-9163-403f-84e2-2dae1e9baa3e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:49.711838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb657b-a31b-4d37-a8ae-89d4836a483b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:49.712474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb657b-a31b-4d37-a8ae-89d4836a483b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:49.712820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb657b-a31b-4d37-a8ae-89d4836a483b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:49.713153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb657b-a31b-4d37-a8ae-89d4836a483b None admin] PATH_INFO: `/v3/users/fd866208c70e43639bb105515ea2921e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:49.714655 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb657b-a31b-4d37-a8ae-89d4836a483b None admin] RBAC: Authorizing `identity:get_user(user_id=fd866208c70e43639bb105515ea2921e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:49.716124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb657b-a31b-4d37-a8ae-89d4836a483b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:49.717920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 72/144] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:49 2022] GET /identity/v3/users/fd866208c70e43639bb105515ea2921e => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:49.724219 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb657b-a31b-4d37-a8ae-89d4836a483b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:49.825561 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:49.843032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb657b-a31b-4d37-a8ae-89d4836a483b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:49.915554 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fafe4f-61d1-4a42-abb6-bcb06a9cb1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:49.916172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fafe4f-61d1-4a42-abb6-bcb06a9cb1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:49.916454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fafe4f-61d1-4a42-abb6-bcb06a9cb1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:49.916728 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fafe4f-61d1-4a42-abb6-bcb06a9cb1b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:49.923137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6593fa12-9236-447f-8c1d-6de2d36b9ab1 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:50.062511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027de4a-5e07-4f4a-baa1-e164bef09bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:50.063163 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027de4a-5e07-4f4a-baa1-e164bef09bf1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:50.063440 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027de4a-5e07-4f4a-baa1-e164bef09bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:50.063703 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c027de4a-5e07-4f4a-baa1-e164bef09bf1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:50.070004 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027de4a-5e07-4f4a-baa1-e164bef09bf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:50.071610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c027de4a-5e07-4f4a-baa1-e164bef09bf1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:50.073798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 73/145] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:58:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 498 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:50.081280 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027de4a-5e07-4f4a-baa1-e164bef09bf1 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:50.081440 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fafe4f-61d1-4a42-abb6-bcb06a9cb1b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:50.082905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fafe4f-61d1-4a42-abb6-bcb06a9cb1b5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:50.214572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:50.284023 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c027de4a-5e07-4f4a-baa1-e164bef09bf1 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:50.286192 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:50.344409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fccc70-c5c4-460a-89ed-ed71d42743e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:50.345406 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fccc70-c5c4-460a-89ed-ed71d42743e3 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:50.345886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fccc70-c5c4-460a-89ed-ed71d42743e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:50.346351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fccc70-c5c4-460a-89ed-ed71d42743e3 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:50.347192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fccc70-c5c4-460a-89ed-ed71d42743e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:50.348817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fccc70-c5c4-460a-89ed-ed71d42743e3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:50.368647 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 73/146] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:58:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 18:58:50.375860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fafe4f-61d1-4a42-abb6-bcb06a9cb1b5 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:50.399679 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 74/147] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 18:58:50 2022] POST /identity/v3/users => generated 283 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 18:58:50.554281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fafe4f-61d1-4a42-abb6-bcb06a9cb1b5 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:50.606355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69c1a90-6cdb-4fef-9481-6e42d257a4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:50.606904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c1a90-6cdb-4fef-9481-6e42d257a4da None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:50.607179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c1a90-6cdb-4fef-9481-6e42d257a4da None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:50.607447 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69c1a90-6cdb-4fef-9481-6e42d257a4da None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:50.614326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c1a90-6cdb-4fef-9481-6e42d257a4da None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:50.615830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69c1a90-6cdb-4fef-9481-6e42d257a4da None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:50.617930 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 74/148] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:50 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:50.624589 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fccc70-c5c4-460a-89ed-ed71d42743e3 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:50.626869 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec55a99-bcb2-48b4-a004-12bfae9996ef None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:50.627148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec55a99-bcb2-48b4-a004-12bfae9996ef None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:50.627418 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec55a99-bcb2-48b4-a004-12bfae9996ef None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:50.629094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 75/149] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:50.646291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb7adf-40f4-42e5-8fda-5ce6e0bc893e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:50.646291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb7adf-40f4-42e5-8fda-5ce6e0bc893e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:50.646655 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb7adf-40f4-42e5-8fda-5ce6e0bc893e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:50.682224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-6dfb7adf-40f4-42e5-8fda-5ce6e0bc893e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:50.684178 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6dfb7adf-40f4-42e5-8fda-5ce6e0bc893e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:50.761739 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:50.802308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 76/150] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:58:50.803713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b022a8-b4f9-47bc-8bf0-dff0ce8bbec2 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:50.804003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b022a8-b4f9-47bc-8bf0-dff0ce8bbec2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:50.804281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b022a8-b4f9-47bc-8bf0-dff0ce8bbec2 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:50.805553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 77/151] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:50.805658 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fccc70-c5c4-460a-89ed-ed71d42743e3 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:50.811749 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1083c1-c251-454e-88fb-bd213c34fa1a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:50.812035 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1083c1-c251-454e-88fb-bd213c34fa1a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:50.812312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1083c1-c251-454e-88fb-bd213c34fa1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:50.832282 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-6b1083c1-c251-454e-88fb-bd213c34fa1a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:50.833507 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6b1083c1-c251-454e-88fb-bd213c34fa1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:50.851754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:50.906925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f6455-75c0-4157-9158-0d7116b1c809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:50.907463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f6455-75c0-4157-9158-0d7116b1c809 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:50.907735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f6455-75c0-4157-9158-0d7116b1c809 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:50.908000 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f6455-75c0-4157-9158-0d7116b1c809 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:50.908726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f6455-75c0-4157-9158-0d7116b1c809 None admin] RBAC: Adding query filter params (role.id=b80a0d1a8d164b53bef5726b92e68676, scope.project.id=7d21b4b8971a47ff8985309015eb53d4, user.id=fd866208c70e43639bb105515ea2921e) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:50.909028 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f6455-75c0-4157-9158-0d7116b1c809 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:50.910501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f6455-75c0-4157-9158-0d7116b1c809 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:50.913348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 78/152] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:50.927492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 75/153] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:58:50 2022] GET /identity/v3/role_assignments?user.id=fd866208c70e43639bb105515ea2921e&scope.project.id=7d21b4b8971a47ff8985309015eb53d4&role.id=b80a0d1a8d164b53bef5726b92e68676 => generated 608 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:51.143378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1083c1-c251-454e-88fb-bd213c34fa1a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:51.249373 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:51.252686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1083c1-c251-454e-88fb-bd213c34fa1a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:51.258381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f6455-75c0-4157-9158-0d7116b1c809 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:51.294042 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44da8be0-3156-4601-9322-44d25ef970c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:51.294673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da8be0-3156-4601-9322-44d25ef970c6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:51.294956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da8be0-3156-4601-9322-44d25ef970c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:51.295227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44da8be0-3156-4601-9322-44d25ef970c6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:51.306437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44da8be0-3156-4601-9322-44d25ef970c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:51.312788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-44da8be0-3156-4601-9322-44d25ef970c6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:58:51.312788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:51.312788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:51.312788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:51.312788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:51.312788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:51.312788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:51.312788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:51.312788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:51.312788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:51.312788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:51.312788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:51.312788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:51.312788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:51.312788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:51.312788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:51.313663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:51.313663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:51.313663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:51.313663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:51.313663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:51.313663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:51.313663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:51.313663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:51.313663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:51.313663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:51.313663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:51.313663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:51.313663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:51.313663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:51.313663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:51.313663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:51.314414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:51.314414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:51.314414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:51.314414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:51.314414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:51.314414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:51.314414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:51.314414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:51.314414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:51.314414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:51.314414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:58:51.314414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:58:51.314414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 79/154] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:58:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:51.314414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86537b03-52b0-42a0-acf1-6231ed3fd037 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:51.315084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86537b03-52b0-42a0-acf1-6231ed3fd037 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:51.315084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86537b03-52b0-42a0-acf1-6231ed3fd037 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:51.320713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 80/155] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:51.320713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e6b31-2890-4e7e-951d-1c05da9402c6 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:51.320713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e6b31-2890-4e7e-951d-1c05da9402c6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:51.320713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e6b31-2890-4e7e-951d-1c05da9402c6 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:51.323401 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 81/156] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:51.324726 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92215b6e-a955-4f5e-a069-45f1e3ef53a7 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:51.325018 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92215b6e-a955-4f5e-a069-45f1e3ef53a7 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:51.325317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92215b6e-a955-4f5e-a069-45f1e3ef53a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:51.345642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-92215b6e-a955-4f5e-a069-45f1e3ef53a7 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:51.347153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-92215b6e-a955-4f5e-a069-45f1e3ef53a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:51.401845 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f6455-75c0-4157-9158-0d7116b1c809 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:51.438316 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 82/157] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:51.439669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d00dd9-fbbb-4494-ae4d-26b903b3932b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:51.439961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d00dd9-fbbb-4494-ae4d-26b903b3932b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:51.440234 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d00dd9-fbbb-4494-ae4d-26b903b3932b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:51.441075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a597e-9935-4d63-84e6-b3ab68d4b48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:51.441750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a597e-9935-4d63-84e6-b3ab68d4b48a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:51.442029 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a597e-9935-4d63-84e6-b3ab68d4b48a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:51.442378 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a597e-9935-4d63-84e6-b3ab68d4b48a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:51.453993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a597e-9935-4d63-84e6-b3ab68d4b48a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:51.459563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-b58a597e-9935-4d63-84e6-b3ab68d4b48a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:58:51.459563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:51.459563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:51.459563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:51.459563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:51.459563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:51.459563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:51.459563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:51.459563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:51.459563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:51.459563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:51.459563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:51.459563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:51.459563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:51.459563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:51.459563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:51.460770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:51.460770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:51.460770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:51.460770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:51.460770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:51.460770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:51.460770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:51.460770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:51.460770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:51.460770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:51.460770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:51.460770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:51.460770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:51.460770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:51.460770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:51.460770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:51.461835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:51.461835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:51.461835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:51.461835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:51.461835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:51.461835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:51.461835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:51.461835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:51.461835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:51.461835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:51.461835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:58:51.461835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:58:51.463801 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 76/158] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 18:58:51.464827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a597e-9935-4d63-84e6-b3ab68d4b48a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:51.475029 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f4d00dd9-fbbb-4494-ae4d-26b903b3932b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:51.476545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f4d00dd9-fbbb-4494-ae4d-26b903b3932b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:51.525471 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:51.558576 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 83/159] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:51.559961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d00dd9-fbbb-4494-ae4d-26b903b3932b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:51.588080 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:51.608601 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a597e-9935-4d63-84e6-b3ab68d4b48a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:51.647480 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af779c9-69b7-4800-b719-a43c53f0d779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:51.648452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af779c9-69b7-4800-b719-a43c53f0d779 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:51.648925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af779c9-69b7-4800-b719-a43c53f0d779 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:51.649423 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af779c9-69b7-4800-b719-a43c53f0d779 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:51.650294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af779c9-69b7-4800-b719-a43c53f0d779 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:51.650788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af779c9-69b7-4800-b719-a43c53f0d779 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:51.652943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af779c9-69b7-4800-b719-a43c53f0d779 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:51.662910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 77/160] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:51.669543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af779c9-69b7-4800-b719-a43c53f0d779 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:51.755691 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d00dd9-fbbb-4494-ae4d-26b903b3932b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:51.796884 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af779c9-69b7-4800-b719-a43c53f0d779 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:51.799045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:51.841940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b1f05-b5e0-4f35-a6ad-1bc6aa773db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:51.842552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b1f05-b5e0-4f35-a6ad-1bc6aa773db0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:51.842833 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b1f05-b5e0-4f35-a6ad-1bc6aa773db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:51.843106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b1f05-b5e0-4f35-a6ad-1bc6aa773db0 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:51.843591 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3df75e-a528-47ff-b3e2-e204743815ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:51.843802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b1f05-b5e0-4f35-a6ad-1bc6aa773db0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:51.844078 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b1f05-b5e0-4f35-a6ad-1bc6aa773db0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:51.844868 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3df75e-a528-47ff-b3e2-e204743815ea None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:51.845439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3df75e-a528-47ff-b3e2-e204743815ea None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:51.845539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b1f05-b5e0-4f35-a6ad-1bc6aa773db0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:51.846026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3df75e-a528-47ff-b3e2-e204743815ea None admin] PATH_INFO: `/v3/users/2d817dbb81e84447ac0c66c6621a6e8f` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:51.848560 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3df75e-a528-47ff-b3e2-e204743815ea None admin] RBAC: Authorizing `identity:get_user(user_id=2d817dbb81e84447ac0c66c6621a6e8f)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:51.850298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3df75e-a528-47ff-b3e2-e204743815ea None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:51.853360 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 78/161] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:51 2022] GET /identity/v3/users/2d817dbb81e84447ac0c66c6621a6e8f => generated 247 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:51.855526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3df75e-a528-47ff-b3e2-e204743815ea None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:51.859193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 84/162] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:58:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:51.863156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b1f05-b5e0-4f35-a6ad-1bc6aa773db0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:51.958311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:51.983405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b1f05-b5e0-4f35-a6ad-1bc6aa773db0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:52.005000 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3df75e-a528-47ff-b3e2-e204743815ea None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:52.028427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252fab3-587d-4540-aa27-0d3beb4baf12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:52.028962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252fab3-587d-4540-aa27-0d3beb4baf12 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:52.029263 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252fab3-587d-4540-aa27-0d3beb4baf12 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:52.029537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252fab3-587d-4540-aa27-0d3beb4baf12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:52.048960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56dd52-e134-491a-b6c8-a54e22220380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:52.050098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56dd52-e134-491a-b6c8-a54e22220380 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:52.050383 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56dd52-e134-491a-b6c8-a54e22220380 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:52.050807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da56dd52-e134-491a-b6c8-a54e22220380 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:52.064171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da56dd52-e134-491a-b6c8-a54e22220380 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:52.076777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-da56dd52-e134-491a-b6c8-a54e22220380 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:58:52.076777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:52.076777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:52.076777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:52.076777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:52.076777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:52.076777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:52.076777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:52.076777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:52.076777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:52.076777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:52.076777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:52.076777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:52.076777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:52.076777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:52.076777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:52.077755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:52.077755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:52.077755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:52.077755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:52.077755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:52.077755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:52.077755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:52.077755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:52.077755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:52.077755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:52.077755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:52.077755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:52.077755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:52.077755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:52.077755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:52.077755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:52.078517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:52.078517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:52.078517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:52.078517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:52.078517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:52.078517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:52.078517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:52.078517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:52.078517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:52.078517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:52.078517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:58:52.078517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:58:52.078517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 79/163] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:52.079149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56dd52-e134-491a-b6c8-a54e22220380 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:52.145556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252fab3-587d-4540-aa27-0d3beb4baf12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:52.145873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:52.147091 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252fab3-587d-4540-aa27-0d3beb4baf12 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:52.279790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da56dd52-e134-491a-b6c8-a54e22220380 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:52.308879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:52.342121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 85/164] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:58:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:58:52.342806 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252fab3-587d-4540-aa27-0d3beb4baf12 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:52.355040 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df25a99-8070-4be3-a0bc-18538b86d7e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:52.355957 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df25a99-8070-4be3-a0bc-18538b86d7e7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:52.356272 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df25a99-8070-4be3-a0bc-18538b86d7e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:52.356647 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df25a99-8070-4be3-a0bc-18538b86d7e7 None admin] PATH_INFO: `/v3/users/0b6cfad5da8e49df8e594be53d842ea5` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:52.362186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df25a99-8070-4be3-a0bc-18538b86d7e7 None admin] RBAC: Authorizing `identity:get_user(user_id=0b6cfad5da8e49df8e594be53d842ea5)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:52.363688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df25a99-8070-4be3-a0bc-18538b86d7e7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:52.365675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 80/165] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:52 2022] GET /identity/v3/users/0b6cfad5da8e49df8e594be53d842ea5 => generated 289 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:52.366463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df25a99-8070-4be3-a0bc-18538b86d7e7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:52.545919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252fab3-587d-4540-aa27-0d3beb4baf12 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:52.558704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:52.562889 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df25a99-8070-4be3-a0bc-18538b86d7e7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:52.606037 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d068a6-cb5b-4e3f-b9fd-feb0300c7938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:52.606904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d068a6-cb5b-4e3f-b9fd-feb0300c7938 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:52.607183 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d068a6-cb5b-4e3f-b9fd-feb0300c7938 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:52.607454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d068a6-cb5b-4e3f-b9fd-feb0300c7938 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:52.624342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d068a6-cb5b-4e3f-b9fd-feb0300c7938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:52.630952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efe98f-cca4-4993-9f58-2fa9205ec1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:52.631567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efe98f-cca4-4993-9f58-2fa9205ec1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:52.631847 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efe98f-cca4-4993-9f58-2fa9205ec1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:52.632120 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efe98f-cca4-4993-9f58-2fa9205ec1b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:52.632780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efe98f-cca4-4993-9f58-2fa9205ec1b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:52.633079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efe98f-cca4-4993-9f58-2fa9205ec1b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:52.634535 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efe98f-cca4-4993-9f58-2fa9205ec1b8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:52.637959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-e8d068a6-cb5b-4e3f-b9fd-feb0300c7938 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:58:52.637959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:52.637959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:52.637959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:52.637959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:52.637959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:52.637959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:52.637959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:52.637959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:52.637959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:52.637959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:52.637959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:52.637959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:52.637959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:52.637959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:52.637959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:52.638808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:52.638808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:52.638808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:52.638808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:52.638808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:52.638808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:52.638808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:52.638808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:52.638808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:52.638808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:52.638808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:52.638808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:52.638808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:52.638808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:52.638808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:52.638808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:52.639668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:52.639668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:52.639668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:52.639668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:52.639668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:52.639668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:52.639668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:52.639668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:52.639668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:52.639668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:52.639668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:58:52.639668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:58:52.639668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 86/166] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 18:58:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:52.640318 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d068a6-cb5b-4e3f-b9fd-feb0300c7938 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:52.653109 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 81/167] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:58:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:52.653816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efe98f-cca4-4993-9f58-2fa9205ec1b8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:52.723123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:52.819928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efe98f-cca4-4993-9f58-2fa9205ec1b8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:52.824690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d068a6-cb5b-4e3f-b9fd-feb0300c7938 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:52.865920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660842cf-b8e1-471e-bae9-08cdde91d5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:52.866995 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660842cf-b8e1-471e-bae9-08cdde91d5fb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:52.867469 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660842cf-b8e1-471e-bae9-08cdde91d5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:52.867973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660842cf-b8e1-471e-bae9-08cdde91d5fb None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:52.869332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660842cf-b8e1-471e-bae9-08cdde91d5fb None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:52.871662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660842cf-b8e1-471e-bae9-08cdde91d5fb None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:52.873925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 87/168] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:52 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:52.875148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660842cf-b8e1-471e-bae9-08cdde91d5fb None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:52.973975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e060b-f699-4069-b6b9-e0f78d0dc406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:52.974575 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e060b-f699-4069-b6b9-e0f78d0dc406 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:52.974851 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e060b-f699-4069-b6b9-e0f78d0dc406 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:52.975120 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903e060b-f699-4069-b6b9-e0f78d0dc406 None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:52.981878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e060b-f699-4069-b6b9-e0f78d0dc406 None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:52.983374 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903e060b-f699-4069-b6b9-e0f78d0dc406 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:52.983476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660842cf-b8e1-471e-bae9-08cdde91d5fb None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:52.989220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 82/169] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:52 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:52.989220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e060b-f699-4069-b6b9-e0f78d0dc406 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:53.015077 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:53.028579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:53.119578 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903e060b-f699-4069-b6b9-e0f78d0dc406 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:53.125895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882789b-43d9-4eb5-9c40-9dbe62c7619c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:53.126530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882789b-43d9-4eb5-9c40-9dbe62c7619c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:53.126814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882789b-43d9-4eb5-9c40-9dbe62c7619c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:53.127089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c882789b-43d9-4eb5-9c40-9dbe62c7619c None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:53.127767 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882789b-43d9-4eb5-9c40-9dbe62c7619c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:53.128072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882789b-43d9-4eb5-9c40-9dbe62c7619c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:53.129575 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c882789b-43d9-4eb5-9c40-9dbe62c7619c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:53.147317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 88/170] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:58:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:53.148272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882789b-43d9-4eb5-9c40-9dbe62c7619c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:53.162539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668249c-76cc-4bb0-a329-cd535b6b2a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:53.163497 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668249c-76cc-4bb0-a329-cd535b6b2a0f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:53.163972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668249c-76cc-4bb0-a329-cd535b6b2a0f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:53.164485 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668249c-76cc-4bb0-a329-cd535b6b2a0f None admin] PATH_INFO: `/v3/users/fd866208c70e43639bb105515ea2921e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:53.166417 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668249c-76cc-4bb0-a329-cd535b6b2a0f None admin] RBAC: Authorizing `identity:get_user(user_id=fd866208c70e43639bb105515ea2921e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:53.168192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668249c-76cc-4bb0-a329-cd535b6b2a0f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:53.170130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 83/171] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:52 2022] GET /identity/v3/users/fd866208c70e43639bb105515ea2921e => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:53.171043 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668249c-76cc-4bb0-a329-cd535b6b2a0f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:53.293424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:53.302411 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668249c-76cc-4bb0-a329-cd535b6b2a0f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:53.343260 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572d044-468a-4208-a54e-705d34afdff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:53.344349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572d044-468a-4208-a54e-705d34afdff0 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:53.344828 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572d044-468a-4208-a54e-705d34afdff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:53.345329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572d044-468a-4208-a54e-705d34afdff0 None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4/users/0b6cfad5da8e49df8e594be53d842ea5/roles/2d35524d275d4bc4b09ce775ddec4963` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:53.348573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572d044-468a-4208-a54e-705d34afdff0 None admin] RBAC: Authorizing `identity:create_grant(project_id=7d21b4b8971a47ff8985309015eb53d4, user_id=0b6cfad5da8e49df8e594be53d842ea5, role_id=2d35524d275d4bc4b09ce775ddec4963)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:53.350293 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572d044-468a-4208-a54e-705d34afdff0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:53.368004 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 84/172] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 18:58:53 2022] PUT /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4/users/0b6cfad5da8e49df8e594be53d842ea5/roles/2d35524d275d4bc4b09ce775ddec4963 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 18:58:53.374041 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572d044-468a-4208-a54e-705d34afdff0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:53.417653 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c882789b-43d9-4eb5-9c40-9dbe62c7619c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:53.477804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7f6fd-bd32-43cf-8c12-1efc66eee899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:53.478445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7f6fd-bd32-43cf-8c12-1efc66eee899 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:53.478741 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7f6fd-bd32-43cf-8c12-1efc66eee899 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:53.479163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7f6fd-bd32-43cf-8c12-1efc66eee899 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:53.485214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7f6fd-bd32-43cf-8c12-1efc66eee899 None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=7d21b4b8971a47ff8985309015eb53d4, user.id=2d817dbb81e84447ac0c66c6621a6e8f) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:53.485214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7f6fd-bd32-43cf-8c12-1efc66eee899 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:53.485214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7f6fd-bd32-43cf-8c12-1efc66eee899 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:53.493491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 89/173] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:58:53 2022] GET /identity/v3/role_assignments?user.id=2d817dbb81e84447ac0c66c6621a6e8f&scope.project.id=7d21b4b8971a47ff8985309015eb53d4&role.id=3580c894dd974d35bc95848b8ba22598 => generated 608 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:53.493491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7f6fd-bd32-43cf-8c12-1efc66eee899 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:53.571908 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572d044-468a-4208-a54e-705d34afdff0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:53.575201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:53.629357 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b215877-9588-4971-adc0-3720e0e99a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:53.630373 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b215877-9588-4971-adc0-3720e0e99a36 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:53.630858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b215877-9588-4971-adc0-3720e0e99a36 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:53.631344 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b215877-9588-4971-adc0-3720e0e99a36 None admin] PATH_INFO: `/v3/users/fd866208c70e43639bb105515ea2921e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:53.632771 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b215877-9588-4971-adc0-3720e0e99a36 None admin] RBAC: Authorizing `identity:get_user(user_id=fd866208c70e43639bb105515ea2921e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:53.634614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b215877-9588-4971-adc0-3720e0e99a36 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:53.637684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 85/174] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:53 2022] GET /identity/v3/users/fd866208c70e43639bb105515ea2921e => generated 275 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:53.639163 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488baa43-be7c-43d0-9439-ebf2fa2f3b77 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:53.639669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488baa43-be7c-43d0-9439-ebf2fa2f3b77 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:53.640141 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488baa43-be7c-43d0-9439-ebf2fa2f3b77 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:53.641713 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 86/175] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:53.650896 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef972a0-8386-47c4-8ff2-f251c99d0c2b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:53.651621 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef972a0-8386-47c4-8ff2-f251c99d0c2b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:53.652139 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef972a0-8386-47c4-8ff2-f251c99d0c2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:53.686150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4ef972a0-8386-47c4-8ff2-f251c99d0c2b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:53.687956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4ef972a0-8386-47c4-8ff2-f251c99d0c2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:53.720696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:53.783143 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7f6fd-bd32-43cf-8c12-1efc66eee899 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:53.785137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 87/176] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:53.787129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef972a0-8386-47c4-8ff2-f251c99d0c2b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:53.882587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2080b19-62f0-4224-9860-b3175e595530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:53.883129 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2080b19-62f0-4224-9860-b3175e595530 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:53.883395 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2080b19-62f0-4224-9860-b3175e595530 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:53.883747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2080b19-62f0-4224-9860-b3175e595530 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:53.905600 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef972a0-8386-47c4-8ff2-f251c99d0c2b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:53.946288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b819bf-d5ac-4273-ad7a-b183d82ace4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:53.947430 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b819bf-d5ac-4273-ad7a-b183d82ace4b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:53.948218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b819bf-d5ac-4273-ad7a-b183d82ace4b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:53.948671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b819bf-d5ac-4273-ad7a-b183d82ace4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:53.977020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:54.035185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b819bf-d5ac-4273-ad7a-b183d82ace4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:54.037837 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b819bf-d5ac-4273-ad7a-b183d82ace4b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:54.074417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2080b19-62f0-4224-9860-b3175e595530 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:54.076011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2080b19-62f0-4224-9860-b3175e595530 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:54.143210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:54.163725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 88/177] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:58:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:58:54.165544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed27e9c-2bf5-461b-a1a6-cb0d1cd26f51 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:54.165987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed27e9c-2bf5-461b-a1a6-cb0d1cd26f51 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:54.166449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed27e9c-2bf5-461b-a1a6-cb0d1cd26f51 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:54.169582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 89/178] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:54.170578 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed27e9c-2bf5-461b-a1a6-cb0d1cd26f51 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:54.243600 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:54.250599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 90/179] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:58:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 761 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:58:54.252311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00e77d-59fd-4349-a53c-337c05396c9d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:54.252802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00e77d-59fd-4349-a53c-337c05396c9d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:54.253311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00e77d-59fd-4349-a53c-337c05396c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:54.271851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7a00e77d-59fd-4349-a53c-337c05396c9d None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:54.273361 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7a00e77d-59fd-4349-a53c-337c05396c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:54.295182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed27e9c-2bf5-461b-a1a6-cb0d1cd26f51 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:54.333130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1c37d-5d02-478d-89c2-d848bc0eec61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:54.333777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1c37d-5d02-478d-89c2-d848bc0eec61 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:54.334047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1c37d-5d02-478d-89c2-d848bc0eec61 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:54.334307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1c37d-5d02-478d-89c2-d848bc0eec61 None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:54.335405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1c37d-5d02-478d-89c2-d848bc0eec61 None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:54.336813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a1c37d-5d02-478d-89c2-d848bc0eec61 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:54.338627 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 90/180] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:54 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:54.339576 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab5415-ef61-4154-b4a0-3fdfb1007bbb None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:54.339844 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab5415-ef61-4154-b4a0-3fdfb1007bbb None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:54.340106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aab5415-ef61-4154-b4a0-3fdfb1007bbb None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:54.341443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 91/181] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:54.341981 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab5415-ef61-4154-b4a0-3fdfb1007bbb None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:54.348753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 91/182] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:54.348753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00e77d-59fd-4349-a53c-337c05396c9d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:54.445577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:54.458397 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aab5415-ef61-4154-b4a0-3fdfb1007bbb None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:54.484779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:54.496471 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00e77d-59fd-4349-a53c-337c05396c9d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:54.499178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda79b5f-9fbc-4cac-9ccb-51d762109da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:54.499875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda79b5f-9fbc-4cac-9ccb-51d762109da1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:54.500264 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda79b5f-9fbc-4cac-9ccb-51d762109da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:54.500603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda79b5f-9fbc-4cac-9ccb-51d762109da1 None admin] PATH_INFO: `/v3/projects/ae6e53e6c2af4bfcab38f81ce10a6f58` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:54.501815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda79b5f-9fbc-4cac-9ccb-51d762109da1 None admin] RBAC: Authorizing `identity:get_project(project_id=ae6e53e6c2af4bfcab38f81ce10a6f58)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:54.503301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda79b5f-9fbc-4cac-9ccb-51d762109da1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:54.504997 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 92/183] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:54 2022] GET /identity/v3/projects/ae6e53e6c2af4bfcab38f81ce10a6f58 => generated 312 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:54.505840 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda79b5f-9fbc-4cac-9ccb-51d762109da1 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:54.534710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023139c-5e28-446b-9fbc-9edeb875bd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:54.535334 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023139c-5e28-446b-9fbc-9edeb875bd5c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:54.535614 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023139c-5e28-446b-9fbc-9edeb875bd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:54.535888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023139c-5e28-446b-9fbc-9edeb875bd5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:54.547232 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023139c-5e28-446b-9fbc-9edeb875bd5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:54.552293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-a023139c-5e28-446b-9fbc-9edeb875bd5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:58:54.552293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:54.552293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:54.552293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:54.552293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:54.552293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:54.552293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:54.552293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:54.552293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:54.552293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:54.552293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:54.552293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:54.552293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:54.552293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:54.552293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:54.552293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:54.553207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:54.553207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:54.553207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:54.553207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:54.553207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:54.553207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:54.553207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:54.553207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:54.553207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:54.553207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:54.553207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:54.553207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:54.553207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:54.553207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:54.553207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:54.553207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:54.554043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:54.554043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:54.554043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:54.554043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:54.554043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:54.554043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:54.554043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:54.554043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:54.554043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:54.554043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:54.554043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:58:54.554043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:58:54.557314 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 92/184] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:58:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:54.558621 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dddac0-4e58-49a0-9f28-b0b37e131086 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:54.558938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dddac0-4e58-49a0-9f28-b0b37e131086 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:54.559223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dddac0-4e58-49a0-9f28-b0b37e131086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:54.579428 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-69dddac0-4e58-49a0-9f28-b0b37e131086 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:54.580675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-69dddac0-4e58-49a0-9f28-b0b37e131086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:54.611871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda79b5f-9fbc-4cac-9ccb-51d762109da1 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:54.647223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c4a97-a1f3-4092-aa31-b8a19694f35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:54.647825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c4a97-a1f3-4092-aa31-b8a19694f35a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:54.648093 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c4a97-a1f3-4092-aa31-b8a19694f35a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:54.648356 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8c4a97-a1f3-4092-aa31-b8a19694f35a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:54.649064 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c4a97-a1f3-4092-aa31-b8a19694f35a None admin] RBAC: Adding query filter params (role.id=470b660efbe04083a803e79d8b647160, scope.project.id=7d21b4b8971a47ff8985309015eb53d4, user.id=fd866208c70e43639bb105515ea2921e) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:54.649387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c4a97-a1f3-4092-aa31-b8a19694f35a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:54.650811 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8c4a97-a1f3-4092-aa31-b8a19694f35a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:54.651448 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 93/185] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:54.653224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dddac0-4e58-49a0-9f28-b0b37e131086 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:54.656798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 93/186] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:58:54 2022] GET /identity/v3/role_assignments?user.id=fd866208c70e43639bb105515ea2921e&scope.project.id=7d21b4b8971a47ff8985309015eb53d4&role.id=470b660efbe04083a803e79d8b647160 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:54.657611 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c4a97-a1f3-4092-aa31-b8a19694f35a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:54.768430 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:54.775454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:54.849532 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dddac0-4e58-49a0-9f28-b0b37e131086 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:54.851504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8c4a97-a1f3-4092-aa31-b8a19694f35a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:54.900706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a01c33-9426-4fa6-af25-d61780138623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:54.901528 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a01c33-9426-4fa6-af25-d61780138623 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:54.901854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a01c33-9426-4fa6-af25-d61780138623 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:54.902218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a01c33-9426-4fa6-af25-d61780138623 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:54.902647 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f755ba3-f1cc-4aa4-addf-85d0cc294c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:54.903086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a01c33-9426-4fa6-af25-d61780138623 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:54.903256 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f755ba3-f1cc-4aa4-addf-85d0cc294c8e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:54.903518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a01c33-9426-4fa6-af25-d61780138623 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:54.903628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f755ba3-f1cc-4aa4-addf-85d0cc294c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:54.903796 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f755ba3-f1cc-4aa4-addf-85d0cc294c8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:54.904526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f755ba3-f1cc-4aa4-addf-85d0cc294c8e None admin] RBAC: Adding query filter params (role.id=b80a0d1a8d164b53bef5726b92e68676, scope.project.id=ae6e53e6c2af4bfcab38f81ce10a6f58, user.id=fd866208c70e43639bb105515ea2921e) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:54.904834 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f755ba3-f1cc-4aa4-addf-85d0cc294c8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:54.905476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a01c33-9426-4fa6-af25-d61780138623 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:54.906376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f755ba3-f1cc-4aa4-addf-85d0cc294c8e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:54.912498 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 94/187] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:58:54 2022] GET /identity/v3/role_assignments?user.id=fd866208c70e43639bb105515ea2921e&scope.project.id=ae6e53e6c2af4bfcab38f81ce10a6f58&role.id=b80a0d1a8d164b53bef5726b92e68676 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:54.914326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f755ba3-f1cc-4aa4-addf-85d0cc294c8e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:54.915832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 94/188] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:54.924207 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a01c33-9426-4fa6-af25-d61780138623 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:55.049833 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f755ba3-f1cc-4aa4-addf-85d0cc294c8e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:55.065231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a01c33-9426-4fa6-af25-d61780138623 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:55.074686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:55.098878 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5fb10-566a-43b6-9516-f853c7b6bb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:55.099926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5fb10-566a-43b6-9516-f853c7b6bb35 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:55.100412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5fb10-566a-43b6-9516-f853c7b6bb35 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:55.100874 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f5fb10-566a-43b6-9516-f853c7b6bb35 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:55.102241 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5fb10-566a-43b6-9516-f853c7b6bb35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:55.103427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:55.104256 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f5fb10-566a-43b6-9516-f853c7b6bb35 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:55.107291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 95/189] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:58:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:55.109720 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5fb10-566a-43b6-9516-f853c7b6bb35 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:55.113908 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146ecfd-2e02-4675-a5ad-fe2d5d749c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:55.114802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146ecfd-2e02-4675-a5ad-fe2d5d749c5e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:55.115095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146ecfd-2e02-4675-a5ad-fe2d5d749c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:55.115985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146ecfd-2e02-4675-a5ad-fe2d5d749c5e None admin] PATH_INFO: `/v3/users/663ae4983a984077b2750d2cdf6ce656` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:55.132626 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146ecfd-2e02-4675-a5ad-fe2d5d749c5e None admin] RBAC: Authorizing `identity:get_user(user_id=663ae4983a984077b2750d2cdf6ce656)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:55.134453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146ecfd-2e02-4675-a5ad-fe2d5d749c5e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:55.136160 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 95/190] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:54 2022] GET /identity/v3/users/663ae4983a984077b2750d2cdf6ce656 => generated 283 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:55.136899 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146ecfd-2e02-4675-a5ad-fe2d5d749c5e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:55.234638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f5fb10-566a-43b6-9516-f853c7b6bb35 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:55.270831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146ecfd-2e02-4675-a5ad-fe2d5d749c5e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:55.272194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce1a8b-d4b3-40bc-a7f6-995427ec23df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:55.272823 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce1a8b-d4b3-40bc-a7f6-995427ec23df None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:55.273104 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce1a8b-d4b3-40bc-a7f6-995427ec23df None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:55.273409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ce1a8b-d4b3-40bc-a7f6-995427ec23df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:55.283859 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ce1a8b-d4b3-40bc-a7f6-995427ec23df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:55.289148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-62ce1a8b-d4b3-40bc-a7f6-995427ec23df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:58:55.289148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:55.289148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:55.289148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:55.289148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:55.289148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:55.289148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:55.289148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:55.289148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:55.289148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:55.289148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:55.289148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:55.289148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:55.289148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:55.289148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:55.289148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:55.290050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:55.290050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:55.290050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:55.290050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:55.290050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:55.290050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:55.290050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:55.290050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:55.290050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:55.290050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:55.290050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:55.290050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:55.290050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:55.290050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:55.290050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:55.290050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:55.290880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:55.290880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:55.290880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:55.290880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:55.290880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:55.290880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:55.290880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:55.290880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:55.290880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:55.290880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:55.290880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:58:55.290880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:58:55.290880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 96/191] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 18:58:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:55.291539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce1a8b-d4b3-40bc-a7f6-995427ec23df None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:55.314689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9a41b-4d71-49b4-bb06-5c8c33a9adef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:55.315382 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9a41b-4d71-49b4-bb06-5c8c33a9adef None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:55.315682 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9a41b-4d71-49b4-bb06-5c8c33a9adef None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:55.315968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9a41b-4d71-49b4-bb06-5c8c33a9adef None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:55.316640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9a41b-4d71-49b4-bb06-5c8c33a9adef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:55.318165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9a41b-4d71-49b4-bb06-5c8c33a9adef None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:55.342285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 96/192] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 18:58:55 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:58:55.342495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:55.343332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9a41b-4d71-49b4-bb06-5c8c33a9adef None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:55.383983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:55.440122 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ce1a8b-d4b3-40bc-a7f6-995427ec23df None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:55.479386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ea4b2-dc9f-4759-bccb-41af3cb7bcf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:55.480126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ea4b2-dc9f-4759-bccb-41af3cb7bcf6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:55.480415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ea4b2-dc9f-4759-bccb-41af3cb7bcf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:55.480734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ea4b2-dc9f-4759-bccb-41af3cb7bcf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:55.488663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9a41b-4d71-49b4-bb06-5c8c33a9adef None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:55.529761 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a1515-be86-4d19-833e-80c85012d387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:55.530381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a1515-be86-4d19-833e-80c85012d387 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:55.530661 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a1515-be86-4d19-833e-80c85012d387 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:55.530935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a1515-be86-4d19-833e-80c85012d387 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:55.531605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a1515-be86-4d19-833e-80c85012d387 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:55.531907 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a1515-be86-4d19-833e-80c85012d387 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:55.534172 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1a1515-be86-4d19-833e-80c85012d387 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:55.543217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 97/193] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:58:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:55.549117 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a1515-be86-4d19-833e-80c85012d387 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:55.575961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ea4b2-dc9f-4759-bccb-41af3cb7bcf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:55.577361 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53ea4b2-dc9f-4759-bccb-41af3cb7bcf6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:55.639305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:55.680078 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:55.692134 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1a1515-be86-4d19-833e-80c85012d387 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:55.714334 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 97/194] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:58:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:58:55.719924 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ea4b2-dc9f-4759-bccb-41af3cb7bcf6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:55.732155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1dc12-199d-4b03-b4ba-1761ab856683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:55.732805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1dc12-199d-4b03-b4ba-1761ab856683 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:55.733098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1dc12-199d-4b03-b4ba-1761ab856683 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:55.733416 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad1dc12-199d-4b03-b4ba-1761ab856683 None admin] PATH_INFO: `/v3/users/663ae4983a984077b2750d2cdf6ce656` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:55.734562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1dc12-199d-4b03-b4ba-1761ab856683 None admin] RBAC: Authorizing `identity:get_user(user_id=663ae4983a984077b2750d2cdf6ce656)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:55.736072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad1dc12-199d-4b03-b4ba-1761ab856683 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:55.737822 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 98/195] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:55 2022] GET /identity/v3/users/663ae4983a984077b2750d2cdf6ce656 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:55.743485 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1dc12-199d-4b03-b4ba-1761ab856683 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:55.841533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ea4b2-dc9f-4759-bccb-41af3cb7bcf6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:55.881636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc71cb4d-8af5-4310-a480-c6632321d5b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:55.882269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad1dc12-199d-4b03-b4ba-1761ab856683 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:55.882401 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71cb4d-8af5-4310-a480-c6632321d5b4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:55.882585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71cb4d-8af5-4310-a480-c6632321d5b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:55.882844 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc71cb4d-8af5-4310-a480-c6632321d5b4 None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:55.884040 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71cb4d-8af5-4310-a480-c6632321d5b4 None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:55.885593 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc71cb4d-8af5-4310-a480-c6632321d5b4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:55.887649 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 98/196] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:55 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:55.892779 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bfce2-25a7-4621-a717-da15822afd50 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:55.893372 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bfce2-25a7-4621-a717-da15822afd50 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:55.893830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bfce2-25a7-4621-a717-da15822afd50 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:55.896290 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 99/197] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 18:58:55.897653 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bfce2-25a7-4621-a717-da15822afd50 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:55.924295 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b6b34-cefa-45ac-9823-d22039d3011e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:55.924994 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:55.925444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b6b34-cefa-45ac-9823-d22039d3011e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:55.925702 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b6b34-cefa-45ac-9823-d22039d3011e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:55.926127 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b6b34-cefa-45ac-9823-d22039d3011e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:55.971353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:56.023405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bfce2-25a7-4621-a717-da15822afd50 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:56.030399 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b6b34-cefa-45ac-9823-d22039d3011e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:56.032399 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b6b34-cefa-45ac-9823-d22039d3011e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:56.061090 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c90bf8b-6bdf-46b3-a5bf-634bd0885f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:56.061755 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90bf8b-6bdf-46b3-a5bf-634bd0885f53 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:56.062025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90bf8b-6bdf-46b3-a5bf-634bd0885f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:56.062289 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c90bf8b-6bdf-46b3-a5bf-634bd0885f53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:56.063007 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90bf8b-6bdf-46b3-a5bf-634bd0885f53 None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=8ab3b89f1290419e93f22ea7eee77da8, user.id=663ae4983a984077b2750d2cdf6ce656) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:56.063302 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90bf8b-6bdf-46b3-a5bf-634bd0885f53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:56.064903 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c90bf8b-6bdf-46b3-a5bf-634bd0885f53 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:56.071438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 100/198] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:58:55 2022] GET /identity/v3/role_assignments?user.id=663ae4983a984077b2750d2cdf6ce656&scope.project.id=8ab3b89f1290419e93f22ea7eee77da8&role.id=3580c894dd974d35bc95848b8ba22598 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:56.072624 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7669e72-ea41-4187-be79-0fb1868bbb89 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:56.073098 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7669e72-ea41-4187-be79-0fb1868bbb89 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:56.073449 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7669e72-ea41-4187-be79-0fb1868bbb89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:56.094008 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d7669e72-ea41-4187-be79-0fb1868bbb89 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:56.095193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d7669e72-ea41-4187-be79-0fb1868bbb89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:56.160149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 99/199] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:58:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 18:58:56.166153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b6b34-cefa-45ac-9823-d22039d3011e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:56.188038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 101/200] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:56.256202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:56.307081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b6b34-cefa-45ac-9823-d22039d3011e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:56.347955 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d5746-d164-4876-becb-75f56dd7d723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:56.348506 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d5746-d164-4876-becb-75f56dd7d723 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:56.348776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d5746-d164-4876-becb-75f56dd7d723 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:56.349061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606d5746-d164-4876-becb-75f56dd7d723 None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:56.350259 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d5746-d164-4876-becb-75f56dd7d723 None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:56.351829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606d5746-d164-4876-becb-75f56dd7d723 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:56.353773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 100/201] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:56 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:56.360451 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7669e72-ea41-4187-be79-0fb1868bbb89 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:56.374529 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:56.479413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7669e72-ea41-4187-be79-0fb1868bbb89 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:56.521536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e2390-a5eb-44fb-84e9-bd87517b0c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:56.522185 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e2390-a5eb-44fb-84e9-bd87517b0c05 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:56.522484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e2390-a5eb-44fb-84e9-bd87517b0c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:56.522768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e2390-a5eb-44fb-84e9-bd87517b0c05 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:56.523621 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479e2390-a5eb-44fb-84e9-bd87517b0c05 None admin] RBAC: Adding query filter params (role.id=470b660efbe04083a803e79d8b647160, scope.project.id=8ab3b89f1290419e93f22ea7eee77da8, user.id=663ae4983a984077b2750d2cdf6ce656) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:56.523972 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479e2390-a5eb-44fb-84e9-bd87517b0c05 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:56.525838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479e2390-a5eb-44fb-84e9-bd87517b0c05 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:56.532249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 102/202] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:58:56 2022] GET /identity/v3/role_assignments?user.id=663ae4983a984077b2750d2cdf6ce656&scope.project.id=8ab3b89f1290419e93f22ea7eee77da8&role.id=470b660efbe04083a803e79d8b647160 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:56.537259 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d5746-d164-4876-becb-75f56dd7d723 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:56.695351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606d5746-d164-4876-becb-75f56dd7d723 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:56.730092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:56.733685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dea9c0-0459-4050-8698-7a97c9a0f20d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:56.734749 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dea9c0-0459-4050-8698-7a97c9a0f20d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:56.735447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dea9c0-0459-4050-8698-7a97c9a0f20d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:56.735734 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dea9c0-0459-4050-8698-7a97c9a0f20d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:56.747486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dea9c0-0459-4050-8698-7a97c9a0f20d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:56.752945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-a8dea9c0-0459-4050-8698-7a97c9a0f20d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:58:56.752945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:56.752945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:56.752945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:56.752945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:56.752945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:56.752945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:56.752945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:56.752945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:56.752945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:56.752945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:56.752945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:56.752945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:56.752945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:56.752945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:56.752945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:56.753846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:56.753846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:56.753846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:56.753846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:56.753846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:56.753846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:56.753846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:56.753846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:56.753846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:56.753846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:56.753846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:56.753846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:56.753846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:56.753846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:56.753846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:56.753846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:56.754622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:56.754622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:56.754622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:56.754622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:56.754622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:56.754622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:56.754622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:56.754622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:56.754622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:56.754622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:56.754622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:58:56.754622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:58:56.754622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 101/203] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:56.761888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e2390-a5eb-44fb-84e9-bd87517b0c05 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:56.881019 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:56.915137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e2390-a5eb-44fb-84e9-bd87517b0c05 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:56.962914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44e470-acbe-4490-96ef-6090d2cc3599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:56.963514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44e470-acbe-4490-96ef-6090d2cc3599 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:56.963781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44e470-acbe-4490-96ef-6090d2cc3599 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:56.964571 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44e470-acbe-4490-96ef-6090d2cc3599 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:56.965239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44e470-acbe-4490-96ef-6090d2cc3599 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:56.965541 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44e470-acbe-4490-96ef-6090d2cc3599 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:56.966893 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d44e470-acbe-4490-96ef-6090d2cc3599 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:56.975611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 103/204] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:58:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:56.981961 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dea9c0-0459-4050-8698-7a97c9a0f20d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:57.086308 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dea9c0-0459-4050-8698-7a97c9a0f20d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:57.121715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288bea4-64cc-4237-a48e-a8899fcb2c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:57.123014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288bea4-64cc-4237-a48e-a8899fcb2c4e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:57.123285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288bea4-64cc-4237-a48e-a8899fcb2c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:57.123550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3288bea4-64cc-4237-a48e-a8899fcb2c4e None admin] PATH_INFO: `/v3/users/0b6cfad5da8e49df8e594be53d842ea5` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:57.124778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288bea4-64cc-4237-a48e-a8899fcb2c4e None admin] RBAC: Authorizing `identity:get_user(user_id=0b6cfad5da8e49df8e594be53d842ea5)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:57.126225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3288bea4-64cc-4237-a48e-a8899fcb2c4e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:57.127785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 102/205] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:56 2022] GET /identity/v3/users/0b6cfad5da8e49df8e594be53d842ea5 => generated 289 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:57.133472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44e470-acbe-4490-96ef-6090d2cc3599 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:57.289154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44e470-acbe-4490-96ef-6090d2cc3599 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:57.337628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94ad68-237e-41a2-b5d4-d7923a2ba242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:57.338233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94ad68-237e-41a2-b5d4-d7923a2ba242 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:57.338505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94ad68-237e-41a2-b5d4-d7923a2ba242 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:57.338791 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a94ad68-237e-41a2-b5d4-d7923a2ba242 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:57.346196 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:57.427303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94ad68-237e-41a2-b5d4-d7923a2ba242 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:57.428753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a94ad68-237e-41a2-b5d4-d7923a2ba242 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:57.557580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 104/206] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:58:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:58:57.566185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288bea4-64cc-4237-a48e-a8899fcb2c4e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:57.652106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:57.715477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3288bea4-64cc-4237-a48e-a8899fcb2c4e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:57.763063 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953d8879-5fd2-4ac8-a58e-9b94f0f3e922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:57.763717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d8879-5fd2-4ac8-a58e-9b94f0f3e922 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:57.763993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d8879-5fd2-4ac8-a58e-9b94f0f3e922 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:57.764261 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953d8879-5fd2-4ac8-a58e-9b94f0f3e922 None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:57.765842 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d8879-5fd2-4ac8-a58e-9b94f0f3e922 None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:57.767292 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953d8879-5fd2-4ac8-a58e-9b94f0f3e922 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:57.769283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 103/207] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:57 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:57.774950 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94ad68-237e-41a2-b5d4-d7923a2ba242 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:57.830551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:57.917359 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a94ad68-237e-41a2-b5d4-d7923a2ba242 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:57.969935 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8c084-d4bd-488c-9caf-b79fdd705341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:57.970582 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8c084-d4bd-488c-9caf-b79fdd705341 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:57.970809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8c084-d4bd-488c-9caf-b79fdd705341 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:57.971084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8c084-d4bd-488c-9caf-b79fdd705341 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:57.971821 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8c084-d4bd-488c-9caf-b79fdd705341 None admin] RBAC: Adding query filter params (role.id=2d35524d275d4bc4b09ce775ddec4963, scope.project.id=7d21b4b8971a47ff8985309015eb53d4, user.id=0b6cfad5da8e49df8e594be53d842ea5) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:57.972128 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8c084-d4bd-488c-9caf-b79fdd705341 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:57.973588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8c084-d4bd-488c-9caf-b79fdd705341 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:57.980222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 105/208] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:58:57 2022] GET /identity/v3/role_assignments?user.id=0b6cfad5da8e49df8e594be53d842ea5&scope.project.id=7d21b4b8971a47ff8985309015eb53d4&role.id=2d35524d275d4bc4b09ce775ddec4963 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:57.998329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebfaeb-b473-4a02-9020-5442be8cb3a2 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:57.998329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebfaeb-b473-4a02-9020-5442be8cb3a2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:57.998329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaebfaeb-b473-4a02-9020-5442be8cb3a2 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:57.998329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 104/209] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:58.003580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2402d-9407-4a01-adf6-da3dafa19db8 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:58.004171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2402d-9407-4a01-adf6-da3dafa19db8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:58.004638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a2402d-9407-4a01-adf6-da3dafa19db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:58.024242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e4a2402d-9407-4a01-adf6-da3dafa19db8 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:58.025962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e4a2402d-9407-4a01-adf6-da3dafa19db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:58.100449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da786fe3-df02-438a-925d-d2e779149194 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:58.101044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da786fe3-df02-438a-925d-d2e779149194 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:58.101564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da786fe3-df02-438a-925d-d2e779149194 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:58.103201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 105/210] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 18:58:58.109390 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdcd81-1e3d-4597-bcad-4d1ce0fc373b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:58.110010 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdcd81-1e3d-4597-bcad-4d1ce0fc373b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:58.110477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdcd81-1e3d-4597-bcad-4d1ce0fc373b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:58.127703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 106/211] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:58.134642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-43cdcd81-1e3d-4597-bcad-4d1ce0fc373b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:58.136827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-43cdcd81-1e3d-4597-bcad-4d1ce0fc373b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:58.216314 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 106/212] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:58:58.420176 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2402d-9407-4a01-adf6-da3dafa19db8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:58.457448 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:58.501772 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdcd81-1e3d-4597-bcad-4d1ce0fc373b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:58.542879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a2402d-9407-4a01-adf6-da3dafa19db8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:58.563841 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:58.590211 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f4808-c0d6-4570-a50a-bce6ea9a4fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:58.591204 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f4808-c0d6-4570-a50a-bce6ea9a4fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:58.591674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f4808-c0d6-4570-a50a-bce6ea9a4fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:58.592133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780f4808-c0d6-4570-a50a-bce6ea9a4fd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:58.606723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780f4808-c0d6-4570-a50a-bce6ea9a4fd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:58.612968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-780f4808-c0d6-4570-a50a-bce6ea9a4fd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:58:58.612968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:58.612968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:58.612968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:58.612968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:58.612968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:58.612968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:58.612968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:58.612968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:58.612968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:58.612968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:58.612968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:58.612968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:58.612968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:58.612968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:58.612968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:58.613847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:58.613847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:58.613847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:58.613847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:58.613847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:58.613847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:58.613847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:58.613847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:58.613847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:58.613847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:58.613847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:58.613847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:58.613847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:58.613847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:58.613847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:58.613847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:58.614613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:58.614613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:58.614613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:58.614613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:58.614613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:58.614613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:58.614613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:58.614613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:58.614613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:58.614613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:58.614613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:58:58.614613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:58:58.617065 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 107/213] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:58.623218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdcd81-1e3d-4597-bcad-4d1ce0fc373b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:58.623690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f4808-c0d6-4570-a50a-bce6ea9a4fd4 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:58.661866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c593a8-0d87-4f8e-a1c3-955a06d69389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:58.662499 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c593a8-0d87-4f8e-a1c3-955a06d69389 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:58.662777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c593a8-0d87-4f8e-a1c3-955a06d69389 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:58.663048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c593a8-0d87-4f8e-a1c3-955a06d69389 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:58.664146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c593a8-0d87-4f8e-a1c3-955a06d69389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:58.665653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c593a8-0d87-4f8e-a1c3-955a06d69389 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:58.667300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 107/214] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:58:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:58.668548 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624dd66-9fda-4ece-a4ed-115e919b467f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:58.668830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624dd66-9fda-4ece-a4ed-115e919b467f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:58.669098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624dd66-9fda-4ece-a4ed-115e919b467f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:58.670377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 108/215] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:58.672387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624dd66-9fda-4ece-a4ed-115e919b467f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:58.726768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:58.738343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780f4808-c0d6-4570-a50a-bce6ea9a4fd4 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:58.776672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c48b2-c807-44d6-ad22-8e04794f975c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:58.777301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c48b2-c807-44d6-ad22-8e04794f975c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:58.777572 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c48b2-c807-44d6-ad22-8e04794f975c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:58.777837 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c48b2-c807-44d6-ad22-8e04794f975c None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:58.778486 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c48b2-c807-44d6-ad22-8e04794f975c None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:58.778777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c48b2-c807-44d6-ad22-8e04794f975c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:58.780129 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c48b2-c807-44d6-ad22-8e04794f975c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:58.783319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624dd66-9fda-4ece-a4ed-115e919b467f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:58.793142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 108/216] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:58:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:58.793142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8754451-998e-4cf4-9185-f5426a7349c8 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:58.793142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8754451-998e-4cf4-9185-f5426a7349c8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:58.793142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8754451-998e-4cf4-9185-f5426a7349c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:58.808533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a8754451-998e-4cf4-9185-f5426a7349c8 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:58.809721 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a8754451-998e-4cf4-9185-f5426a7349c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:58.811621 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:58.825302 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e11d57-786c-4ee8-a19a-1342398dbcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:58.825983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e11d57-786c-4ee8-a19a-1342398dbcc9 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:58.826264 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e11d57-786c-4ee8-a19a-1342398dbcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:58.826533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e11d57-786c-4ee8-a19a-1342398dbcc9 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:58.827179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e11d57-786c-4ee8-a19a-1342398dbcc9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:58.828609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e11d57-786c-4ee8-a19a-1342398dbcc9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:58.861218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 109/217] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 18:58:58 2022] POST /identity/v3/users => generated 297 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:58:58.861218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e11d57-786c-4ee8-a19a-1342398dbcc9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:58.897880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 109/218] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:58.992179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e11d57-786c-4ee8-a19a-1342398dbcc9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:59.024493 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83158ad-975b-4c7c-b9f4-e15642d24b62 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:59.025004 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83158ad-975b-4c7c-b9f4-e15642d24b62 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:59.025112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83158ad-975b-4c7c-b9f4-e15642d24b62 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:59.026804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 110/219] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:58:59.032195 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2edb72-29cb-451f-92be-5a8db79959c1 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:59.032483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2edb72-29cb-451f-92be-5a8db79959c1 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:59.032744 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2edb72-29cb-451f-92be-5a8db79959c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:59.035358 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03762d2-e515-4ef9-ba78-f8144728587d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:59.036214 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03762d2-e515-4ef9-ba78-f8144728587d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:59.036659 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03762d2-e515-4ef9-ba78-f8144728587d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:59.037098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03762d2-e515-4ef9-ba78-f8144728587d None admin] PATH_INFO: `/v3/users/a982cfac711849c1a9cc61a96b4eb2de` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:59.052753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03762d2-e515-4ef9-ba78-f8144728587d None admin] RBAC: Authorizing `identity:get_user(user_id=a982cfac711849c1a9cc61a96b4eb2de)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:59.054535 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03762d2-e515-4ef9-ba78-f8144728587d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:59.054691 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-db2edb72-29cb-451f-92be-5a8db79959c1 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:58:59.055860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-db2edb72-29cb-451f-92be-5a8db79959c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:58:59.057090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 110/220] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:58 2022] GET /identity/v3/users/a982cfac711849c1a9cc61a96b4eb2de => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:59.064024 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03762d2-e515-4ef9-ba78-f8144728587d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:59.110087 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:59.147001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 111/221] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:58:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:58:59.175413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2edb72-29cb-451f-92be-5a8db79959c1 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:59.192561 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:59.195287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03762d2-e515-4ef9-ba78-f8144728587d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:59.241005 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3d62c-2bad-4aa9-b3ab-9b59797abb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:59.241685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3d62c-2bad-4aa9-b3ab-9b59797abb7b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:59.241951 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3d62c-2bad-4aa9-b3ab-9b59797abb7b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:59.242292 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3d62c-2bad-4aa9-b3ab-9b59797abb7b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:59.311833 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2edb72-29cb-451f-92be-5a8db79959c1 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:59.335172 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3d62c-2bad-4aa9-b3ab-9b59797abb7b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:59.337748 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd3d62c-2bad-4aa9-b3ab-9b59797abb7b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:59.350485 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfefecd-8b11-4309-a4cc-be88f530a25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:59.351175 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfefecd-8b11-4309-a4cc-be88f530a25f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:59.351454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfefecd-8b11-4309-a4cc-be88f530a25f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:59.351722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfefecd-8b11-4309-a4cc-be88f530a25f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:59.367473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfefecd-8b11-4309-a4cc-be88f530a25f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:59.375120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-fbfefecd-8b11-4309-a4cc-be88f530a25f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:58:59.375120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:59.375120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:59.375120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:59.375120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:59.375120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:59.375120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:59.375120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:59.375120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:59.375120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:59.375120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:59.375120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:59.375120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:59.375120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:59.375120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:59.375120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:59.376003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:59.376003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:59.376003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:59.376003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:59.376003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:59.376003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:59.376003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:59.376003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:59.376003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:59.376003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:59.376003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:59.376003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:59.376003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:59.376003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:59.376003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:59.376003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:59.376778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:59.376778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:59.376778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:59.376778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:59.376778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:59.376778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:59.376778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:59.376778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:59.376778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:59.376778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:59.376778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:58:59.376778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:58:59.376778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 112/222] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:58:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 18:58:59.387535 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfefecd-8b11-4309-a4cc-be88f530a25f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:59.394943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:59.469543 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:59.476779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 111/223] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:58:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 18:58:59.477744 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3d62c-2bad-4aa9-b3ab-9b59797abb7b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:59.508466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfefecd-8b11-4309-a4cc-be88f530a25f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:59.549718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68b6d6-c89d-46a9-8306-1d44a7c10e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:59.550332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68b6d6-c89d-46a9-8306-1d44a7c10e63 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:59.550614 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68b6d6-c89d-46a9-8306-1d44a7c10e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:59.550886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68b6d6-c89d-46a9-8306-1d44a7c10e63 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:59.551549 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68b6d6-c89d-46a9-8306-1d44a7c10e63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:59.551853 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68b6d6-c89d-46a9-8306-1d44a7c10e63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:59.553243 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68b6d6-c89d-46a9-8306-1d44a7c10e63 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:59.561962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 113/224] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:58:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:58:59.562772 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68b6d6-c89d-46a9-8306-1d44a7c10e63 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:59.605831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3d62c-2bad-4aa9-b3ab-9b59797abb7b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:59.643173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d2ac2-c767-47cb-a3c8-7075c8c98338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:59.643831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d2ac2-c767-47cb-a3c8-7075c8c98338 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:59.644123 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d2ac2-c767-47cb-a3c8-7075c8c98338 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:59.644402 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d2ac2-c767-47cb-a3c8-7075c8c98338 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:59.647447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:59.657621 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d2ac2-c767-47cb-a3c8-7075c8c98338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:58:59.662615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-ed1d2ac2-c767-47cb-a3c8-7075c8c98338 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:58:59.662615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:58:59.662615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:58:59.662615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:58:59.662615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:58:59.662615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:58:59.662615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:58:59.662615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:58:59.662615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:58:59.662615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:58:59.662615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:58:59.662615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:58:59.662615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:58:59.662615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:58:59.662615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:58:59.662615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:58:59.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:58:59.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:58:59.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:58:59.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:58:59.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:58:59.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:58:59.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:58:59.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:58:59.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:58:59.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:58:59.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:58:59.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:58:59.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:58:59.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:58:59.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:58:59.663672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:58:59.664674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:58:59.664674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:58:59.664674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:58:59.664674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:58:59.664674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:58:59.664674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:58:59.664674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:58:59.664674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:58:59.664674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:58:59.664674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:58:59.664674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:58:59.664674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:58:59.665982 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 112/225] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 18:58:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:58:59.666823 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d2ac2-c767-47cb-a3c8-7075c8c98338 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:59.706135 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68b6d6-c89d-46a9-8306-1d44a7c10e63 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:59.749443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f57aa-139d-422e-a358-f2570e55a2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:59.751002 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f57aa-139d-422e-a358-f2570e55a2a0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:59.751356 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f57aa-139d-422e-a358-f2570e55a2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:59.751699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f57aa-139d-422e-a358-f2570e55a2a0 None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:59.752981 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f57aa-139d-422e-a358-f2570e55a2a0 None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:59.754530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f57aa-139d-422e-a358-f2570e55a2a0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:59.757404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 114/226] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:59 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:59.758385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f57aa-139d-422e-a358-f2570e55a2a0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:59.762431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:59.787500 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d2ac2-c767-47cb-a3c8-7075c8c98338 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:59.832185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886295de-87f5-48dd-a417-3b73e7c29530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:59.833205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886295de-87f5-48dd-a417-3b73e7c29530 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:59.833555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886295de-87f5-48dd-a417-3b73e7c29530 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:59.833890 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886295de-87f5-48dd-a417-3b73e7c29530 None admin] PATH_INFO: `/v3/users/663ae4983a984077b2750d2cdf6ce656` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:59.835674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886295de-87f5-48dd-a417-3b73e7c29530 None admin] RBAC: Authorizing `identity:get_user(user_id=663ae4983a984077b2750d2cdf6ce656)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:59.839080 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886295de-87f5-48dd-a417-3b73e7c29530 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:59.841424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 113/227] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:58:59 2022] GET /identity/v3/users/663ae4983a984077b2750d2cdf6ce656 => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:59.842194 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886295de-87f5-48dd-a417-3b73e7c29530 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:59.888832 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f57aa-139d-422e-a358-f2570e55a2a0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:58:59.915050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:58:59.924519 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313aaf74-61d2-4554-b4a1-6dd28aafb03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:58:59.925128 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aaf74-61d2-4554-b4a1-6dd28aafb03f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:58:59.925446 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aaf74-61d2-4554-b4a1-6dd28aafb03f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:58:59.925717 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313aaf74-61d2-4554-b4a1-6dd28aafb03f None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:58:59.926435 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313aaf74-61d2-4554-b4a1-6dd28aafb03f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:58:59.926741 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313aaf74-61d2-4554-b4a1-6dd28aafb03f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:58:59.928483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313aaf74-61d2-4554-b4a1-6dd28aafb03f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:58:59.937468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 115/228] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:58:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:58:59.938304 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313aaf74-61d2-4554-b4a1-6dd28aafb03f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:58:59.966252 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886295de-87f5-48dd-a417-3b73e7c29530 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:00.003066 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:00.008441 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910073fb-7f5f-44fc-81c7-4c06c40619b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:00.009523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910073fb-7f5f-44fc-81c7-4c06c40619b7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:00.009796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910073fb-7f5f-44fc-81c7-4c06c40619b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:00.010060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910073fb-7f5f-44fc-81c7-4c06c40619b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:00.010781 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910073fb-7f5f-44fc-81c7-4c06c40619b7 None admin] RBAC: Adding query filter params (role.id=b80a0d1a8d164b53bef5726b92e68676, scope.project.id=8ab3b89f1290419e93f22ea7eee77da8, user.id=a982cfac711849c1a9cc61a96b4eb2de) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:00.011192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910073fb-7f5f-44fc-81c7-4c06c40619b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:00.012906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910073fb-7f5f-44fc-81c7-4c06c40619b7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:00.019262 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 114/229] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:58:59 2022] GET /identity/v3/role_assignments?user.id=a982cfac711849c1a9cc61a96b4eb2de&scope.project.id=8ab3b89f1290419e93f22ea7eee77da8&role.id=b80a0d1a8d164b53bef5726b92e68676 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:00.020361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910073fb-7f5f-44fc-81c7-4c06c40619b7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:00.046097 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313aaf74-61d2-4554-b4a1-6dd28aafb03f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:00.083079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c727819-edc5-478a-9e69-e5fe7e6f5feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:00.083728 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c727819-edc5-478a-9e69-e5fe7e6f5feb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:00.084006 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c727819-edc5-478a-9e69-e5fe7e6f5feb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:00.084280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c727819-edc5-478a-9e69-e5fe7e6f5feb None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:00.085470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c727819-edc5-478a-9e69-e5fe7e6f5feb None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:00.086926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c727819-edc5-478a-9e69-e5fe7e6f5feb None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:00.088650 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 116/230] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:58:59 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:00.095071 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c727819-edc5-478a-9e69-e5fe7e6f5feb None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:00.148941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910073fb-7f5f-44fc-81c7-4c06c40619b7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:00.212248 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:00.229337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe5cb6-ab4c-49b1-a8d7-e3dc9f1b71eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:00.231979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe5cb6-ab4c-49b1-a8d7-e3dc9f1b71eb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:00.233015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe5cb6-ab4c-49b1-a8d7-e3dc9f1b71eb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:00.234170 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fe5cb6-ab4c-49b1-a8d7-e3dc9f1b71eb None admin] PATH_INFO: `/v3/users/663ae4983a984077b2750d2cdf6ce656` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:00.238346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe5cb6-ab4c-49b1-a8d7-e3dc9f1b71eb None admin] RBAC: Authorizing `identity:get_user(user_id=663ae4983a984077b2750d2cdf6ce656)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:00.240620 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fe5cb6-ab4c-49b1-a8d7-e3dc9f1b71eb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:00.242861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 115/231] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:59:00 2022] GET /identity/v3/users/663ae4983a984077b2750d2cdf6ce656 => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:00.248905 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe5cb6-ab4c-49b1-a8d7-e3dc9f1b71eb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:00.277537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c727819-edc5-478a-9e69-e5fe7e6f5feb None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:00.315430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:00.325212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7346b-9260-4a17-8e31-e04f5e0c4537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:00.325212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7346b-9260-4a17-8e31-e04f5e0c4537 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:00.325212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7346b-9260-4a17-8e31-e04f5e0c4537 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:00.325212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7346b-9260-4a17-8e31-e04f5e0c4537 None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8/users/663ae4983a984077b2750d2cdf6ce656/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:00.325574 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7346b-9260-4a17-8e31-e04f5e0c4537 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ab3b89f1290419e93f22ea7eee77da8, user_id=663ae4983a984077b2750d2cdf6ce656, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:00.326296 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7346b-9260-4a17-8e31-e04f5e0c4537 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:00.348955 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 117/232] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 18:59:00 2022] PUT /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8/users/663ae4983a984077b2750d2cdf6ce656/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 18:59:00.397615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fe5cb6-ab4c-49b1-a8d7-e3dc9f1b71eb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:00.435499 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53334f2-1be5-4192-848f-63ff4895c78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:00.436547 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53334f2-1be5-4192-848f-63ff4895c78a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:00.437018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53334f2-1be5-4192-848f-63ff4895c78a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:00.437507 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53334f2-1be5-4192-848f-63ff4895c78a None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:00.438954 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53334f2-1be5-4192-848f-63ff4895c78a None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:00.440609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53334f2-1be5-4192-848f-63ff4895c78a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:00.442792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 116/233] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:00 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:00.449822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7346b-9260-4a17-8e31-e04f5e0c4537 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:00.585803 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7346b-9260-4a17-8e31-e04f5e0c4537 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:00.620422 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b26f2-b8cd-43f0-8a8e-0d1499dc14e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:00.621084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b26f2-b8cd-43f0-8a8e-0d1499dc14e7 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:00.621392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b26f2-b8cd-43f0-8a8e-0d1499dc14e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:00.621668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b26f2-b8cd-43f0-8a8e-0d1499dc14e7 None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8/users/663ae4983a984077b2750d2cdf6ce656/roles/470b660efbe04083a803e79d8b647160` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:00.624366 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b26f2-b8cd-43f0-8a8e-0d1499dc14e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ab3b89f1290419e93f22ea7eee77da8, user_id=663ae4983a984077b2750d2cdf6ce656, role_id=470b660efbe04083a803e79d8b647160)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:00.625823 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b26f2-b8cd-43f0-8a8e-0d1499dc14e7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:00.642033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 118/234] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 18:59:00 2022] PUT /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8/users/663ae4983a984077b2750d2cdf6ce656/roles/470b660efbe04083a803e79d8b647160 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 18:59:01.327396 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d33a82-b478-4bbe-a827-506c5d7eb46f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:01.328385 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d33a82-b478-4bbe-a827-506c5d7eb46f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:01.328880 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d33a82-b478-4bbe-a827-506c5d7eb46f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:01.330735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 117/235] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:01.335782 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4ffd9-d439-4a6b-979a-60bce2d2665a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:01.336178 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4ffd9-d439-4a6b-979a-60bce2d2665a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:01.336390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4ffd9-d439-4a6b-979a-60bce2d2665a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:01.355611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-5fe4ffd9-d439-4a6b-979a-60bce2d2665a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:01.361915 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5fe4ffd9-d439-4a6b-979a-60bce2d2665a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:01.453727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:01.493274 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 119/236] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:01.665455 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0d2fb-d7f7-49ea-9bc2-1a01902235da None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:01.665455 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0d2fb-d7f7-49ea-9bc2-1a01902235da None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:01.665455 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd0d2fb-d7f7-49ea-9bc2-1a01902235da None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:01.665455 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 118/237] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:01.671365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c77c34-5d33-4ab2-8013-18b2f61df8ab None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:01.671712 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c77c34-5d33-4ab2-8013-18b2f61df8ab None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:01.671989 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c77c34-5d33-4ab2-8013-18b2f61df8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:01.690563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e3c77c34-5d33-4ab2-8013-18b2f61df8ab None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:01.691738 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e3c77c34-5d33-4ab2-8013-18b2f61df8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:01.783902 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 120/238] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:01.968362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0d2fb-d7f7-49ea-9bc2-1a01902235da None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:02.044865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:02.102796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd0d2fb-d7f7-49ea-9bc2-1a01902235da None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:02.146022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486cf0d3-90f2-4229-9d46-6ff678366492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:02.146991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cf0d3-90f2-4229-9d46-6ff678366492 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:02.147518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cf0d3-90f2-4229-9d46-6ff678366492 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:02.148034 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486cf0d3-90f2-4229-9d46-6ff678366492 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:02.159197 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486cf0d3-90f2-4229-9d46-6ff678366492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:02.171894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-486cf0d3-90f2-4229-9d46-6ff678366492 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:59:02.171894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:02.171894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:02.171894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:02.171894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:02.171894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:02.171894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:02.171894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:02.171894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:02.171894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:02.171894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:02.171894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:02.171894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:02.171894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:02.171894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:02.171894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:02.172824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:02.172824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:02.172824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:02.172824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:02.172824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:02.172824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:02.172824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:02.172824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:02.172824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:02.172824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:02.172824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:02.172824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:02.172824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:02.172824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:02.172824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:02.172824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:02.173656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:02.173656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:02.173656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:02.173656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:02.173656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:02.173656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:02.173656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:02.173656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:02.173656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:02.173656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:02.173656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:59:02.173656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:02.176466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 119/239] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:02.182377 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c77c34-5d33-4ab2-8013-18b2f61df8ab None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:02.319254 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486cf0d3-90f2-4229-9d46-6ff678366492 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:02.358206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c77c34-5d33-4ab2-8013-18b2f61df8ab None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:02.385812 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:02.432718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc44b8-3b82-4902-93c2-bc3cc6420411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:02.433887 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc44b8-3b82-4902-93c2-bc3cc6420411 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:02.434164 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc44b8-3b82-4902-93c2-bc3cc6420411 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:02.434435 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dc44b8-3b82-4902-93c2-bc3cc6420411 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:02.435098 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc44b8-3b82-4902-93c2-bc3cc6420411 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:02.435399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc44b8-3b82-4902-93c2-bc3cc6420411 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:02.436782 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dc44b8-3b82-4902-93c2-bc3cc6420411 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:02.445680 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 121/240] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:02.451982 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc44b8-3b82-4902-93c2-bc3cc6420411 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:02.526148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486cf0d3-90f2-4229-9d46-6ff678366492 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:02.560623 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dc44b8-3b82-4902-93c2-bc3cc6420411 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:02.576936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:02.599541 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d9a7d-fe2e-4db3-af7d-69c99efa2866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:02.600205 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d9a7d-fe2e-4db3-af7d-69c99efa2866 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:02.600446 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d9a7d-fe2e-4db3-af7d-69c99efa2866 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:02.600719 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66d9a7d-fe2e-4db3-af7d-69c99efa2866 None admin] PATH_INFO: `/v3/users/8288f9bc86f244a3879ef7de7e3b7625` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:02.602002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffddaec0-017d-42cf-88d1-5f7532bfbf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:02.602636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddaec0-017d-42cf-88d1-5f7532bfbf0c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:02.602911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddaec0-017d-42cf-88d1-5f7532bfbf0c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:02.603184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddaec0-017d-42cf-88d1-5f7532bfbf0c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:02.606609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffddaec0-017d-42cf-88d1-5f7532bfbf0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:02.608071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffddaec0-017d-42cf-88d1-5f7532bfbf0c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:02.609702 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 120/241] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:02.615372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffddaec0-017d-42cf-88d1-5f7532bfbf0c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:02.615933 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d9a7d-fe2e-4db3-af7d-69c99efa2866 None admin] RBAC: Authorizing `identity:get_user(user_id=8288f9bc86f244a3879ef7de7e3b7625)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:02.617529 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66d9a7d-fe2e-4db3-af7d-69c99efa2866 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:02.624228 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 122/242] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:59:02 2022] GET /identity/v3/users/8288f9bc86f244a3879ef7de7e3b7625 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:02.625478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d9a7d-fe2e-4db3-af7d-69c99efa2866 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:02.659331 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:02.735206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66d9a7d-fe2e-4db3-af7d-69c99efa2866 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:02.756487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffddaec0-017d-42cf-88d1-5f7532bfbf0c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:02.773611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358d864-3a1a-454d-86d5-ca73e6654ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:02.774543 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358d864-3a1a-454d-86d5-ca73e6654ab2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:02.774984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358d864-3a1a-454d-86d5-ca73e6654ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:02.775398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7358d864-3a1a-454d-86d5-ca73e6654ab2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:02.796342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e72ae-418a-4348-94d2-0cecbd412c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:02.796950 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e72ae-418a-4348-94d2-0cecbd412c9d None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:02.797250 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e72ae-418a-4348-94d2-0cecbd412c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:02.797557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e72ae-418a-4348-94d2-0cecbd412c9d None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:02.798161 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e72ae-418a-4348-94d2-0cecbd412c9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:02.799593 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e72ae-418a-4348-94d2-0cecbd412c9d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:02.829327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 121/243] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 18:59:02 2022] POST /identity/v3/users => generated 293 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 18:59:02.829327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d14560-2245-4ed4-9f88-0ced42a9f7e6 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:02.829327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d14560-2245-4ed4-9f88-0ced42a9f7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:02.829327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d14560-2245-4ed4-9f88-0ced42a9f7e6 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:02.829327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 122/244] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:02.831750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a110b-2b5d-4caa-b4bd-1c00e7d96442 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:02.832039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a110b-2b5d-4caa-b4bd-1c00e7d96442 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:02.832301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86a110b-2b5d-4caa-b4bd-1c00e7d96442 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:02.833722 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 123/245] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:02.834846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a131a6-f903-4017-a855-594cc61ee82f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:02.835107 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a131a6-f903-4017-a855-594cc61ee82f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:02.835375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a131a6-f903-4017-a855-594cc61ee82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:02.859021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358d864-3a1a-454d-86d5-ca73e6654ab2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:02.860822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7358d864-3a1a-454d-86d5-ca73e6654ab2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:02.861878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f2a131a6-f903-4017-a855-594cc61ee82f None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:02.863114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f2a131a6-f903-4017-a855-594cc61ee82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:02.899137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:02.935759 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:02.967107 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 124/246] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:02.974836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c3106-142a-4291-a806-e2ae2ec3338a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:02.975116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c3106-142a-4291-a806-e2ae2ec3338a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:02.975389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488c3106-142a-4291-a806-e2ae2ec3338a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:02.994735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 123/247] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 18:59:02.996766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358d864-3a1a-454d-86d5-ca73e6654ab2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:03.014216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-488c3106-142a-4291-a806-e2ae2ec3338a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:03.015486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-488c3106-142a-4291-a806-e2ae2ec3338a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:03.100025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 125/248] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:03.101626 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58927b97-5e36-47dc-8f47-cf7f7fffb343 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:03.101973 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58927b97-5e36-47dc-8f47-cf7f7fffb343 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:03.102247 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58927b97-5e36-47dc-8f47-cf7f7fffb343 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:03.106075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 126/249] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:03.111929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf9448-b919-4a31-9df4-d6345cb5eb76 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:03.112210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf9448-b919-4a31-9df4-d6345cb5eb76 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:03.112475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdf9448-b919-4a31-9df4-d6345cb5eb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:03.119516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7358d864-3a1a-454d-86d5-ca73e6654ab2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:03.137279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4bdf9448-b919-4a31-9df4-d6345cb5eb76 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:03.138478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4bdf9448-b919-4a31-9df4-d6345cb5eb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:03.152694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:03.158718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a331ec6-a694-4740-838f-7810307c4bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:03.159728 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a331ec6-a694-4740-838f-7810307c4bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:03.160201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a331ec6-a694-4740-838f-7810307c4bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:03.160724 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a331ec6-a694-4740-838f-7810307c4bc5 None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:03.162106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a331ec6-a694-4740-838f-7810307c4bc5 None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:03.163893 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a331ec6-a694-4740-838f-7810307c4bc5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:03.166076 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 124/250] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:02 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:03.172582 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a331ec6-a694-4740-838f-7810307c4bc5 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:03.222908 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 127/251] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:03.286733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf9448-b919-4a31-9df4-d6345cb5eb76 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:03.304571 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a331ec6-a694-4740-838f-7810307c4bc5 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:03.343555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:03.354603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b09cb-a7c3-42ef-a0cf-997e8856bb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:03.355673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b09cb-a7c3-42ef-a0cf-997e8856bb5b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:03.355959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b09cb-a7c3-42ef-a0cf-997e8856bb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:03.356216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b09cb-a7c3-42ef-a0cf-997e8856bb5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:03.356943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b09cb-a7c3-42ef-a0cf-997e8856bb5b None admin] RBAC: Adding query filter params (role.id=2d35524d275d4bc4b09ce775ddec4963, scope.project.id=8ab3b89f1290419e93f22ea7eee77da8, user.id=8288f9bc86f244a3879ef7de7e3b7625) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:03.357292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b09cb-a7c3-42ef-a0cf-997e8856bb5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:03.358743 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b09cb-a7c3-42ef-a0cf-997e8856bb5b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:03.365383 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 125/252] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:59:03 2022] GET /identity/v3/role_assignments?user.id=8288f9bc86f244a3879ef7de7e3b7625&scope.project.id=8ab3b89f1290419e93f22ea7eee77da8&role.id=2d35524d275d4bc4b09ce775ddec4963 => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:03.423336 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdf9448-b919-4a31-9df4-d6345cb5eb76 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:03.458657 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b09cb-a7c3-42ef-a0cf-997e8856bb5b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:03.469449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aab77b-65ae-49c6-b417-e270a81dd7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:03.470346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aab77b-65ae-49c6-b417-e270a81dd7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:03.470822 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aab77b-65ae-49c6-b417-e270a81dd7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:03.471223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aab77b-65ae-49c6-b417-e270a81dd7f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:03.483778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aab77b-65ae-49c6-b417-e270a81dd7f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:03.489049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-08aab77b-65ae-49c6-b417-e270a81dd7f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:03.489049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:03.489049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:03.489049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:03.489049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:03.489049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:03.489049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:03.489049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:03.489049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:03.489049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:03.489049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:03.489049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:03.489049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:03.489049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:03.489049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:03.489049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:03.490025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:03.490025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:03.490025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:03.490025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:03.490025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:03.490025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:03.490025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:03.490025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:03.490025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:03.490025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:03.490025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:03.490025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:03.490025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:03.490025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:03.490025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:03.490025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:03.490802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:03.490802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:03.490802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:03.490802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:03.490802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:03.490802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:03.490802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:03.490802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:03.490802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:03.490802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:03.490802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:03.490802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:03.490802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 128/253] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 18:59:03.497609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aab77b-65ae-49c6-b417-e270a81dd7f4 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:03.567490 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:03.616907 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b09cb-a7c3-42ef-a0cf-997e8856bb5b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:03.654700 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:03.661576 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aab77b-65ae-49c6-b417-e270a81dd7f4 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:03.691560 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac878691-4ca0-4b5b-bc25-98ad0c5140b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:03.692199 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac878691-4ca0-4b5b-bc25-98ad0c5140b6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:03.692472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac878691-4ca0-4b5b-bc25-98ad0c5140b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:03.692739 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac878691-4ca0-4b5b-bc25-98ad0c5140b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:03.712466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac878691-4ca0-4b5b-bc25-98ad0c5140b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:03.721904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-ac878691-4ca0-4b5b-bc25-98ad0c5140b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:59:03.721904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:03.721904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:03.721904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:03.721904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:03.721904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:03.721904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:03.721904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:03.721904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:03.721904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:03.721904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:03.721904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:03.721904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:03.721904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:03.721904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:03.721904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:03.722795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:03.722795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:03.722795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:03.722795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:03.722795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:03.722795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:03.722795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:03.722795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:03.722795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:03.722795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:03.722795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:03.722795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:03.722795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:03.722795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:03.722795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:03.722795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:03.723566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:03.723566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:03.723566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:03.723566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:03.723566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:03.723566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:03.723566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:03.723566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:03.723566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:03.723566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:03.723566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:59:03.723566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:03.724181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 126/254] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:59:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 18:59:03.724735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac878691-4ca0-4b5b-bc25-98ad0c5140b6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:03.734312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a52ce5-ab84-4577-af61-94517838cadf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:03.734928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a52ce5-ab84-4577-af61-94517838cadf None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:03.735204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a52ce5-ab84-4577-af61-94517838cadf None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:03.735523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a52ce5-ab84-4577-af61-94517838cadf None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:03.736187 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a52ce5-ab84-4577-af61-94517838cadf None admin] RBAC: Adding query filter params (name=member) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:03.736482 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a52ce5-ab84-4577-af61-94517838cadf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:03.737925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a52ce5-ab84-4577-af61-94517838cadf None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:03.753354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 129/255] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:03.754526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a52ce5-ab84-4577-af61-94517838cadf None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:03.845549 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac878691-4ca0-4b5b-bc25-98ad0c5140b6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:03.871382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:03.883858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20dfb70-f58b-43d7-b259-5ada8060b275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:03.884487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dfb70-f58b-43d7-b259-5ada8060b275 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:03.884765 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dfb70-f58b-43d7-b259-5ada8060b275 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:03.885033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dfb70-f58b-43d7-b259-5ada8060b275 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:03.895825 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20dfb70-f58b-43d7-b259-5ada8060b275 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:03.900762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-e20dfb70-f58b-43d7-b259-5ada8060b275 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:59:03.900762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:03.900762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:03.900762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:03.900762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:03.900762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:03.900762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:03.900762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:03.900762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:03.900762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:03.900762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:03.900762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:03.900762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:03.900762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:03.900762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:03.900762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:03.902181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:03.902181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:03.902181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:03.902181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:03.902181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:03.902181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:03.902181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:03.902181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:03.902181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:03.902181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:03.902181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:03.902181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:03.902181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:03.902181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:03.902181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:03.902181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:03.902956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:03.902956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:03.902956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:03.902956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:03.902956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:03.902956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:03.902956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:03.902956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:03.902956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:03.902956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:03.902956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:59:03.902956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:03.902956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 127/256] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 18:59:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:03.903608 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20dfb70-f58b-43d7-b259-5ada8060b275 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:04.003162 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a52ce5-ab84-4577-af61-94517838cadf None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:04.015146 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20dfb70-f58b-43d7-b259-5ada8060b275 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:04.070848 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b2f29-1379-41d1-bc4e-0d92d0cec40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:04.071458 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b2f29-1379-41d1-bc4e-0d92d0cec40b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:04.071737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b2f29-1379-41d1-bc4e-0d92d0cec40b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:04.072001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940b2f29-1379-41d1-bc4e-0d92d0cec40b None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:04.072656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b2f29-1379-41d1-bc4e-0d92d0cec40b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:04.072952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b2f29-1379-41d1-bc4e-0d92d0cec40b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:04.074441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927bab0-dbc4-4e08-b4de-c1c1535c94b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:04.075081 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927bab0-dbc4-4e08-b4de-c1c1535c94b1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:04.075184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940b2f29-1379-41d1-bc4e-0d92d0cec40b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:04.075367 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927bab0-dbc4-4e08-b4de-c1c1535c94b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:04.075640 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927bab0-dbc4-4e08-b4de-c1c1535c94b1 None admin] PATH_INFO: `/v3/users/a982cfac711849c1a9cc61a96b4eb2de` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:04.077859 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927bab0-dbc4-4e08-b4de-c1c1535c94b1 None admin] RBAC: Authorizing `identity:get_user(user_id=a982cfac711849c1a9cc61a96b4eb2de)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:04.079289 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927bab0-dbc4-4e08-b4de-c1c1535c94b1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:04.080878 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 128/257] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:59:03 2022] GET /identity/v3/users/a982cfac711849c1a9cc61a96b4eb2de => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:04.081671 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927bab0-dbc4-4e08-b4de-c1c1535c94b1 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:04.090699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 130/258] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:04.091402 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b2f29-1379-41d1-bc4e-0d92d0cec40b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:04.131247 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:04.183354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:04.236158 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940b2f29-1379-41d1-bc4e-0d92d0cec40b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:04.270575 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde13072-c111-4064-8947-6b43ce659fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:04.271211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde13072-c111-4064-8947-6b43ce659fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:04.271489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde13072-c111-4064-8947-6b43ce659fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:04.271764 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde13072-c111-4064-8947-6b43ce659fd1 None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:04.272878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde13072-c111-4064-8947-6b43ce659fd1 None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:04.274584 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde13072-c111-4064-8947-6b43ce659fd1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:04.276526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 131/259] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:04 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:04.277971 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde13072-c111-4064-8947-6b43ce659fd1 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:04.318911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927bab0-dbc4-4e08-b4de-c1c1535c94b1 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:04.389562 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33381b71-8be2-4240-9ab1-f846f1e91f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:04.390193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33381b71-8be2-4240-9ab1-f846f1e91f7a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:04.390446 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33381b71-8be2-4240-9ab1-f846f1e91f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:04.390617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33381b71-8be2-4240-9ab1-f846f1e91f7a None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:04.391269 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33381b71-8be2-4240-9ab1-f846f1e91f7a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:04.391566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33381b71-8be2-4240-9ab1-f846f1e91f7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:04.395009 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33381b71-8be2-4240-9ab1-f846f1e91f7a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:04.411871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 129/260] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:04.415084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33381b71-8be2-4240-9ab1-f846f1e91f7a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:04.423299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:04.441145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde13072-c111-4064-8947-6b43ce659fd1 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:04.511974 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0061f3-bb22-45a0-9cfb-38e281abdc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:04.515361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0061f3-bb22-45a0-9cfb-38e281abdc66 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:04.516003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0061f3-bb22-45a0-9cfb-38e281abdc66 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:04.517147 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0061f3-bb22-45a0-9cfb-38e281abdc66 None admin] PATH_INFO: `/v3/users/663ae4983a984077b2750d2cdf6ce656` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:04.520517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0061f3-bb22-45a0-9cfb-38e281abdc66 None admin] RBAC: Authorizing `identity:get_user(user_id=663ae4983a984077b2750d2cdf6ce656)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:04.524595 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0061f3-bb22-45a0-9cfb-38e281abdc66 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:04.528053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 132/261] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:59:04 2022] GET /identity/v3/users/663ae4983a984077b2750d2cdf6ce656 => generated 283 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:04.535883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0061f3-bb22-45a0-9cfb-38e281abdc66 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:04.617890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33381b71-8be2-4240-9ab1-f846f1e91f7a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:04.637328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:04.662810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be5251-d419-42c2-9c39-704c38dff94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:04.664038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be5251-d419-42c2-9c39-704c38dff94c None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:04.664519 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be5251-d419-42c2-9c39-704c38dff94c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:04.665005 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be5251-d419-42c2-9c39-704c38dff94c None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8/users/a982cfac711849c1a9cc61a96b4eb2de/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:04.668406 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be5251-d419-42c2-9c39-704c38dff94c None admin] RBAC: Authorizing `identity:create_grant(project_id=8ab3b89f1290419e93f22ea7eee77da8, user_id=a982cfac711849c1a9cc61a96b4eb2de, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:04.670322 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be5251-d419-42c2-9c39-704c38dff94c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:04.686359 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 130/262] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 18:59:04 2022] PUT /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8/users/a982cfac711849c1a9cc61a96b4eb2de/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 18:59:04.687683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be5251-d419-42c2-9c39-704c38dff94c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:04.800996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0061f3-bb22-45a0-9cfb-38e281abdc66 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:04.812301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be5251-d419-42c2-9c39-704c38dff94c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:04.852651 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314611d2-cdfd-4778-88cf-c158cc842e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:04.853868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314611d2-cdfd-4778-88cf-c158cc842e08 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:04.854301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314611d2-cdfd-4778-88cf-c158cc842e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:04.854762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b34d1-5d03-4c9b-be46-82af475535f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:04.855033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314611d2-cdfd-4778-88cf-c158cc842e08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:04.855580 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b34d1-5d03-4c9b-be46-82af475535f7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:04.855954 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b34d1-5d03-4c9b-be46-82af475535f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:04.856341 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b34d1-5d03-4c9b-be46-82af475535f7 None admin] PATH_INFO: `/v3/users/663ae4983a984077b2750d2cdf6ce656` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:04.857616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b34d1-5d03-4c9b-be46-82af475535f7 None admin] RBAC: Authorizing `identity:get_user(user_id=663ae4983a984077b2750d2cdf6ce656)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:04.859149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b34d1-5d03-4c9b-be46-82af475535f7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:04.861027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 133/263] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:59:04 2022] GET /identity/v3/users/663ae4983a984077b2750d2cdf6ce656 => generated 283 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:04.866514 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b34d1-5d03-4c9b-be46-82af475535f7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:04.896832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:04.909629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:04.938346 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314611d2-cdfd-4778-88cf-c158cc842e08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:04.940935 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314611d2-cdfd-4778-88cf-c158cc842e08 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:05.023838 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b34d1-5d03-4c9b-be46-82af475535f7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:05.069289 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fdf64-af15-4444-b409-70ff3800f7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:05.070438 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fdf64-af15-4444-b409-70ff3800f7cb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:05.071115 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fdf64-af15-4444-b409-70ff3800f7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:05.071542 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239fdf64-af15-4444-b409-70ff3800f7cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:05.081215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 131/264] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:05.083606 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314611d2-cdfd-4778-88cf-c158cc842e08 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:05.189656 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:05.235879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fdf64-af15-4444-b409-70ff3800f7cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:05.237399 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239fdf64-af15-4444-b409-70ff3800f7cb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:05.258095 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314611d2-cdfd-4778-88cf-c158cc842e08 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:05.301601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be977548-fd20-4d1b-8558-2d83119d0a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:05.302241 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be977548-fd20-4d1b-8558-2d83119d0a8f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:05.302518 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be977548-fd20-4d1b-8558-2d83119d0a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:05.303043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be977548-fd20-4d1b-8558-2d83119d0a8f None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:05.304233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be977548-fd20-4d1b-8558-2d83119d0a8f None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:05.305762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be977548-fd20-4d1b-8558-2d83119d0a8f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:05.308921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 132/265] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:05 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:05.313795 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:05.315075 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be977548-fd20-4d1b-8558-2d83119d0a8f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:05.506333 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 134/266] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 640 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 18:59:05.508003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be977548-fd20-4d1b-8558-2d83119d0a8f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:05.512233 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fdf64-af15-4444-b409-70ff3800f7cb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:05.577343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6604a1c-6246-408f-b09f-96368e527f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:05.577950 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6604a1c-6246-408f-b09f-96368e527f8b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:05.578227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6604a1c-6246-408f-b09f-96368e527f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:05.578494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6604a1c-6246-408f-b09f-96368e527f8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:05.579219 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6604a1c-6246-408f-b09f-96368e527f8b None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=8ab3b89f1290419e93f22ea7eee77da8, user.id=663ae4983a984077b2750d2cdf6ce656) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:05.579522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6604a1c-6246-408f-b09f-96368e527f8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:05.580949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6604a1c-6246-408f-b09f-96368e527f8b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:05.594868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 133/267] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:59:05 2022] GET /identity/v3/role_assignments?user.id=663ae4983a984077b2750d2cdf6ce656&scope.project.id=8ab3b89f1290419e93f22ea7eee77da8&role.id=3580c894dd974d35bc95848b8ba22598 => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:05.672830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239fdf64-af15-4444-b409-70ff3800f7cb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:05.704345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:05.715703 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ae77d2-4feb-4985-b19a-c93ab9b89bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:05.716347 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae77d2-4feb-4985-b19a-c93ab9b89bcb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:05.716876 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae77d2-4feb-4985-b19a-c93ab9b89bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:05.717165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ae77d2-4feb-4985-b19a-c93ab9b89bcb None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:05.718318 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae77d2-4feb-4985-b19a-c93ab9b89bcb None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:05.719770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ae77d2-4feb-4985-b19a-c93ab9b89bcb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:05.721769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 135/268] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:05 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:05.730932 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6604a1c-6246-408f-b09f-96368e527f8b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:05.740389 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:05.846183 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a883778-5a32-4a3d-8cc8-d9ce575931f0 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:05.846678 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a883778-5a32-4a3d-8cc8-d9ce575931f0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:05.846678 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a883778-5a32-4a3d-8cc8-d9ce575931f0 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:05.849301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 136/269] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:05.853841 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d8a89-b12f-408e-91f0-3f89ce222a32 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:05.854428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d8a89-b12f-408e-91f0-3f89ce222a32 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:05.855465 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d8a89-b12f-408e-91f0-3f89ce222a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:05.872761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6604a1c-6246-408f-b09f-96368e527f8b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:05.879223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-681d8a89-b12f-408e-91f0-3f89ce222a32 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:05.885901 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-681d8a89-b12f-408e-91f0-3f89ce222a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:05.938879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20d577-4da4-4c1e-af78-aeb9a53faec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:05.939901 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20d577-4da4-4c1e-af78-aeb9a53faec1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:05.940411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20d577-4da4-4c1e-af78-aeb9a53faec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:05.940880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20d577-4da4-4c1e-af78-aeb9a53faec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:05.941846 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20d577-4da4-4c1e-af78-aeb9a53faec1 None admin] RBAC: Adding query filter params (role.id=470b660efbe04083a803e79d8b647160, scope.project.id=8ab3b89f1290419e93f22ea7eee77da8, user.id=663ae4983a984077b2750d2cdf6ce656) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:05.942529 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20d577-4da4-4c1e-af78-aeb9a53faec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:05.946106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20d577-4da4-4c1e-af78-aeb9a53faec1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:05.957646 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 134/270] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:59:05 2022] GET /identity/v3/role_assignments?user.id=663ae4983a984077b2750d2cdf6ce656&scope.project.id=8ab3b89f1290419e93f22ea7eee77da8&role.id=470b660efbe04083a803e79d8b647160 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:06.001345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 137/271] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:59:06.018983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3354a54-8df8-46b9-b986-dc96bc823721 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:06.019489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b27c1a-6cb2-4c00-a71d-1728cb394f10 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:06.019776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b27c1a-6cb2-4c00-a71d-1728cb394f10 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:06.020052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b27c1a-6cb2-4c00-a71d-1728cb394f10 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:06.020299 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3354a54-8df8-46b9-b986-dc96bc823721 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:06.020775 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3354a54-8df8-46b9-b986-dc96bc823721 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:06.021726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 138/272] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:06.022723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 135/273] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:06.027120 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320e535-f3d7-4164-acdd-aa6cde568691 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:06.027418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320e535-f3d7-4164-acdd-aa6cde568691 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:06.027690 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b320e535-f3d7-4164-acdd-aa6cde568691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:06.033800 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ca8ec-9554-4c44-8ea9-4aed7722aabd None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:06.034342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ca8ec-9554-4c44-8ea9-4aed7722aabd None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:06.034864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ca8ec-9554-4c44-8ea9-4aed7722aabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:06.057336 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e62ca8ec-9554-4c44-8ea9-4aed7722aabd None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:06.057634 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b320e535-f3d7-4164-acdd-aa6cde568691 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:06.058883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b320e535-f3d7-4164-acdd-aa6cde568691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:06.059277 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e62ca8ec-9554-4c44-8ea9-4aed7722aabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:06.154378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:06.161975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 139/274] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:06.164889 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 136/275] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:06.332453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320e535-f3d7-4164-acdd-aa6cde568691 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:06.390959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:06.438400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b320e535-f3d7-4164-acdd-aa6cde568691 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:06.470282 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ca8ec-9554-4c44-8ea9-4aed7722aabd None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:06.477864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197bfed3-19f8-4023-8c23-c13ca72d305d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:06.478526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bfed3-19f8-4023-8c23-c13ca72d305d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:06.478802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bfed3-19f8-4023-8c23-c13ca72d305d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:06.479128 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bfed3-19f8-4023-8c23-c13ca72d305d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:06.491776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197bfed3-19f8-4023-8c23-c13ca72d305d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:06.498340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-197bfed3-19f8-4023-8c23-c13ca72d305d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:59:06.498340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:06.498340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:06.498340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:06.498340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:06.498340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:06.498340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:06.498340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:06.498340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:06.498340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:06.498340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:06.498340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:06.498340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:06.498340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:06.498340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:06.498340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:06.499215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:06.499215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:06.499215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:06.499215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:06.499215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:06.499215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:06.499215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:06.499215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:06.499215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:06.499215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:06.499215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:06.499215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:06.499215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:06.499215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:06.499215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:06.499215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:06.500110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:06.500110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:06.500110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:06.500110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:06.500110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:06.500110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:06.500110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:06.500110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:06.500110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:06.500110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:06.500110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:59:06.500110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:06.500110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 140/276] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:06.501247 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197bfed3-19f8-4023-8c23-c13ca72d305d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:06.592388 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ca8ec-9554-4c44-8ea9-4aed7722aabd None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:06.608110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197bfed3-19f8-4023-8c23-c13ca72d305d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:06.622768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:06.633556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796a391-2973-4e67-af4c-47e9e3d1cce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:06.634224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796a391-2973-4e67-af4c-47e9e3d1cce5 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:06.634445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796a391-2973-4e67-af4c-47e9e3d1cce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:06.634710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9796a391-2973-4e67-af4c-47e9e3d1cce5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:06.635790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796a391-2973-4e67-af4c-47e9e3d1cce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:06.637803 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9796a391-2973-4e67-af4c-47e9e3d1cce5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:06.639528 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 137/277] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:06.644448 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796a391-2973-4e67-af4c-47e9e3d1cce5 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:06.644667 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393a574-cdef-4662-ba00-a7d650472d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:06.645503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393a574-cdef-4662-ba00-a7d650472d7e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:06.645780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393a574-cdef-4662-ba00-a7d650472d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:06.646049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2393a574-cdef-4662-ba00-a7d650472d7e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:06.659231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2393a574-cdef-4662-ba00-a7d650472d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:06.664119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-2393a574-cdef-4662-ba00-a7d650472d7e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:06.664119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:06.664119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:06.664119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:06.664119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:06.664119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:06.664119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:06.664119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:06.664119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:06.664119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:06.664119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:06.664119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:06.664119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:06.664119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:06.664119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:06.664119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:06.664965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:06.664965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:06.664965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:06.664965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:06.664965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:06.664965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:06.664965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:06.664965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:06.664965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:06.664965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:06.664965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:06.664965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:06.664965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:06.664965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:06.664965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:06.664965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:06.665788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:06.665788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:06.665788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:06.665788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:06.665788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:06.665788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:06.665788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:06.665788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:06.665788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:06.665788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:06.665788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:06.665788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:06.665788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 141/278] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:06.666425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393a574-cdef-4662-ba00-a7d650472d7e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:06.719326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:06.756089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9796a391-2973-4e67-af4c-47e9e3d1cce5 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:06.774477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2393a574-cdef-4662-ba00-a7d650472d7e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:06.798527 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf10c58-35b3-4164-a45d-d482f2eec69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:06.799411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf10c58-35b3-4164-a45d-d482f2eec69c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:06.799825 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf10c58-35b3-4164-a45d-d482f2eec69c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:06.800224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf10c58-35b3-4164-a45d-d482f2eec69c None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:06.801239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf10c58-35b3-4164-a45d-d482f2eec69c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:06.801721 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf10c58-35b3-4164-a45d-d482f2eec69c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:06.803965 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf10c58-35b3-4164-a45d-d482f2eec69c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:06.809828 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71791bd9-574a-4424-8b94-5f92c09fe665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:06.810436 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71791bd9-574a-4424-8b94-5f92c09fe665 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:06.811180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71791bd9-574a-4424-8b94-5f92c09fe665 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:06.811456 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71791bd9-574a-4424-8b94-5f92c09fe665 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:06.812106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71791bd9-574a-4424-8b94-5f92c09fe665 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:06.813573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71791bd9-574a-4424-8b94-5f92c09fe665 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:06.813810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 138/279] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:06.820204 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf10c58-35b3-4164-a45d-d482f2eec69c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:06.849000 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 142/280] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 18:59:06 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:59:06.849000 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71791bd9-574a-4424-8b94-5f92c09fe665 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:06.894801 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:06.935990 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf10c58-35b3-4164-a45d-d482f2eec69c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:06.975210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:06.983552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bd05b-a627-4aae-a6ef-5cdf64405dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:06.984644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bd05b-a627-4aae-a6ef-5cdf64405dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:06.985210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bd05b-a627-4aae-a6ef-5cdf64405dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:06.985514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bd05b-a627-4aae-a6ef-5cdf64405dd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:06.986430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065bd05b-a627-4aae-a6ef-5cdf64405dd2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:06.986744 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065bd05b-a627-4aae-a6ef-5cdf64405dd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:06.988934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065bd05b-a627-4aae-a6ef-5cdf64405dd2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:06.990721 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71791bd9-574a-4424-8b94-5f92c09fe665 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:07.001373 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 139/281] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:07.006604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bd05b-a627-4aae-a6ef-5cdf64405dd2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:07.040605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaabbb1-a8cc-4cb9-b9a4-230bccdd9549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:07.041457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaabbb1-a8cc-4cb9-b9a4-230bccdd9549 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:07.041715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaabbb1-a8cc-4cb9-b9a4-230bccdd9549 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:07.042347 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaabbb1-a8cc-4cb9-b9a4-230bccdd9549 None admin] PATH_INFO: `/v3/users/8288f9bc86f244a3879ef7de7e3b7625` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:07.043754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaabbb1-a8cc-4cb9-b9a4-230bccdd9549 None admin] RBAC: Authorizing `identity:get_user(user_id=8288f9bc86f244a3879ef7de7e3b7625)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:07.045788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaabbb1-a8cc-4cb9-b9a4-230bccdd9549 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:07.047398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 143/282] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:59:06 2022] GET /identity/v3/users/8288f9bc86f244a3879ef7de7e3b7625 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:07.052895 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaabbb1-a8cc-4cb9-b9a4-230bccdd9549 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:07.138942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bd05b-a627-4aae-a6ef-5cdf64405dd2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:07.181030 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb20e97-10ec-4af8-9328-09ae6fb1a402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:07.181806 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb20e97-10ec-4af8-9328-09ae6fb1a402 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:07.182090 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb20e97-10ec-4af8-9328-09ae6fb1a402 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:07.182412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb20e97-10ec-4af8-9328-09ae6fb1a402 None admin] PATH_INFO: `/v3/users/45f5041aca054b99ad52b126be30ee1b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:07.194085 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:07.197585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaabbb1-a8cc-4cb9-b9a4-230bccdd9549 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:07.198032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb20e97-10ec-4af8-9328-09ae6fb1a402 None admin] RBAC: Authorizing `identity:get_user(user_id=45f5041aca054b99ad52b126be30ee1b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:07.199710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb20e97-10ec-4af8-9328-09ae6fb1a402 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:07.202675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 140/283] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:59:07 2022] GET /identity/v3/users/45f5041aca054b99ad52b126be30ee1b => generated 293 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:07.204066 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d74a6a-4842-4916-be87-f45259c4b34c None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:07.204569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d74a6a-4842-4916-be87-f45259c4b34c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:07.205027 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d74a6a-4842-4916-be87-f45259c4b34c None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:07.206723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 141/284] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:07.209975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d74a6a-4842-4916-be87-f45259c4b34c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:07.250952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc01cd-fbc5-42b5-b5e0-5310578007ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:07.251901 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc01cd-fbc5-42b5-b5e0-5310578007ac None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:07.252189 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc01cd-fbc5-42b5-b5e0-5310578007ac None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:07.252467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc01cd-fbc5-42b5-b5e0-5310578007ac None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:07.253647 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc01cd-fbc5-42b5-b5e0-5310578007ac None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:07.255088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc01cd-fbc5-42b5-b5e0-5310578007ac None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:07.256748 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 144/285] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:07 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:07.258095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095aab9-e7f7-4408-a3b5-b9194eab5be8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:07.258311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095aab9-e7f7-4408-a3b5-b9194eab5be8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:07.258585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095aab9-e7f7-4408-a3b5-b9194eab5be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:07.280547 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:07.281615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8095aab9-e7f7-4408-a3b5-b9194eab5be8 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:07.283254 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8095aab9-e7f7-4408-a3b5-b9194eab5be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:07.336486 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d74a6a-4842-4916-be87-f45259c4b34c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:07.365588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 145/286] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:07.366220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095aab9-e7f7-4408-a3b5-b9194eab5be8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:07.377277 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64835bfd-7a60-4eb4-ab9d-c0ef31a39681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:07.378224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64835bfd-7a60-4eb4-ab9d-c0ef31a39681 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:07.378736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64835bfd-7a60-4eb4-ab9d-c0ef31a39681 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:07.379206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64835bfd-7a60-4eb4-ab9d-c0ef31a39681 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:07.380160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64835bfd-7a60-4eb4-ab9d-c0ef31a39681 None admin] RBAC: Adding query filter params (role.id=b80a0d1a8d164b53bef5726b92e68676, user.id=45f5041aca054b99ad52b126be30ee1b, scope.system=all) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:07.380665 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64835bfd-7a60-4eb4-ab9d-c0ef31a39681 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:07.382419 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64835bfd-7a60-4eb4-ab9d-c0ef31a39681 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:07.388253 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 142/287] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Aug 31 18:59:07 2022] GET /identity/v3/role_assignments?user.id=45f5041aca054b99ad52b126be30ee1b&scope.system=all&role.id=b80a0d1a8d164b53bef5726b92e68676 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:07.543886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:07.551339 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095aab9-e7f7-4408-a3b5-b9194eab5be8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:07.593967 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2f426-7293-4007-aaf3-d1b87de418a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:07.594636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2f426-7293-4007-aaf3-d1b87de418a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:07.594913 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2f426-7293-4007-aaf3-d1b87de418a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:07.595188 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d2f426-7293-4007-aaf3-d1b87de418a2 None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8/users/8288f9bc86f244a3879ef7de7e3b7625/roles/2d35524d275d4bc4b09ce775ddec4963` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:07.597111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2f426-7293-4007-aaf3-d1b87de418a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8ab3b89f1290419e93f22ea7eee77da8, user_id=8288f9bc86f244a3879ef7de7e3b7625, role_id=2d35524d275d4bc4b09ce775ddec4963)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:07.598586 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d2f426-7293-4007-aaf3-d1b87de418a2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:07.626798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 146/288] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 18:59:07 2022] PUT /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8/users/8288f9bc86f244a3879ef7de7e3b7625/roles/2d35524d275d4bc4b09ce775ddec4963 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 18:59:07.694047 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64835bfd-7a60-4eb4-ab9d-c0ef31a39681 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:07.810920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64835bfd-7a60-4eb4-ab9d-c0ef31a39681 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:07.826956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:07.849563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6da04-155a-48c7-82bf-5d41090c9f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:07.850194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6da04-155a-48c7-82bf-5d41090c9f28 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:07.850474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6da04-155a-48c7-82bf-5d41090c9f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:07.850747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a6da04-155a-48c7-82bf-5d41090c9f28 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:07.861450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a6da04-155a-48c7-82bf-5d41090c9f28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:07.866423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-c0a6da04-155a-48c7-82bf-5d41090c9f28 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:07.866423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:07.866423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:07.866423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:07.866423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:07.866423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:07.866423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:07.866423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:07.866423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:07.866423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:07.866423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:07.866423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:07.866423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:07.866423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:07.866423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:07.866423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:07.867278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:07.867278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:07.867278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:07.867278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:07.867278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:07.867278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:07.867278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:07.867278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:07.867278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:07.867278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:07.867278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:07.867278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:07.867278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:07.867278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:07.867278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:07.867278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:07.868113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:07.868113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:07.868113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:07.868113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:07.868113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:07.868113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:07.868113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:07.868113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:07.868113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:07.868113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:07.868113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:07.868113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:07.870183 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 143/289] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:07.874197 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2f426-7293-4007-aaf3-d1b87de418a2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:07.981925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d2f426-7293-4007-aaf3-d1b87de418a2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:08.017079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea7787-be79-4437-97ab-2a58abeb1c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:08.017734 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea7787-be79-4437-97ab-2a58abeb1c80 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:08.018018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea7787-be79-4437-97ab-2a58abeb1c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:08.018291 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ea7787-be79-4437-97ab-2a58abeb1c80 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:08.018952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea7787-be79-4437-97ab-2a58abeb1c80 None admin] RBAC: Adding query filter params (name=member) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:08.019306 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea7787-be79-4437-97ab-2a58abeb1c80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:08.020682 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ea7787-be79-4437-97ab-2a58abeb1c80 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:08.033298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 147/290] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:08.035691 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6da04-155a-48c7-82bf-5d41090c9f28 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:08.139445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a6da04-155a-48c7-82bf-5d41090c9f28 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:08.174538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7592b9-3d91-409a-821d-c3b32da6054e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:08.175175 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7592b9-3d91-409a-821d-c3b32da6054e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:08.175454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7592b9-3d91-409a-821d-c3b32da6054e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:08.175723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7592b9-3d91-409a-821d-c3b32da6054e None admin] PATH_INFO: `/v3/users/a982cfac711849c1a9cc61a96b4eb2de` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:08.176846 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7592b9-3d91-409a-821d-c3b32da6054e None admin] RBAC: Authorizing `identity:get_user(user_id=a982cfac711849c1a9cc61a96b4eb2de)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:08.178374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7592b9-3d91-409a-821d-c3b32da6054e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:08.184652 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 144/291] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:59:08 2022] GET /identity/v3/users/a982cfac711849c1a9cc61a96b4eb2de => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:08.186061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea7787-be79-4437-97ab-2a58abeb1c80 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:08.206532 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:08.289937 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ea7787-be79-4437-97ab-2a58abeb1c80 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:08.324568 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62817c30-d617-4bac-b2e4-c3236511b30b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:08.325178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62817c30-d617-4bac-b2e4-c3236511b30b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:08.325479 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62817c30-d617-4bac-b2e4-c3236511b30b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:08.325749 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62817c30-d617-4bac-b2e4-c3236511b30b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:08.422330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62817c30-d617-4bac-b2e4-c3236511b30b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:08.423708 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62817c30-d617-4bac-b2e4-c3236511b30b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:08.450879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:08.543273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 148/292] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:08.548643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7592b9-3d91-409a-821d-c3b32da6054e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:08.618096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:08.689097 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7592b9-3d91-409a-821d-c3b32da6054e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:08.722999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0ba5aa-1944-484d-9fbf-0285277deda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:08.723633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ba5aa-1944-484d-9fbf-0285277deda4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:08.723898 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ba5aa-1944-484d-9fbf-0285277deda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:08.724172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0ba5aa-1944-484d-9fbf-0285277deda4 None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:08.725324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ba5aa-1944-484d-9fbf-0285277deda4 None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:08.726848 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0ba5aa-1944-484d-9fbf-0285277deda4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:08.728642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 145/293] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:08 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:08.735025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62817c30-d617-4bac-b2e4-c3236511b30b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:08.857627 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62817c30-d617-4bac-b2e4-c3236511b30b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:08.882033 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:08.907307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972aa6d-7403-45e8-bb7b-7e87fb114799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:08.907920 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972aa6d-7403-45e8-bb7b-7e87fb114799 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:08.908199 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972aa6d-7403-45e8-bb7b-7e87fb114799 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:08.908464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972aa6d-7403-45e8-bb7b-7e87fb114799 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:08.909229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972aa6d-7403-45e8-bb7b-7e87fb114799 None admin] RBAC: Adding query filter params (role.id=b80a0d1a8d164b53bef5726b92e68676, scope.project.id=8ab3b89f1290419e93f22ea7eee77da8, user.id=a982cfac711849c1a9cc61a96b4eb2de) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:08.909513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972aa6d-7403-45e8-bb7b-7e87fb114799 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:08.910930 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972aa6d-7403-45e8-bb7b-7e87fb114799 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:08.922278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 149/294] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:59:08 2022] GET /identity/v3/role_assignments?user.id=a982cfac711849c1a9cc61a96b4eb2de&scope.project.id=8ab3b89f1290419e93f22ea7eee77da8&role.id=b80a0d1a8d164b53bef5726b92e68676 => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:09.377396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53481a-9cfa-424a-9153-ff063014ccc8 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:09.378073 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53481a-9cfa-424a-9153-ff063014ccc8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:09.378561 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53481a-9cfa-424a-9153-ff063014ccc8 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:09.380421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 146/295] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:09.385443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fbde1-3a41-464e-ac07-9d31845a08bc None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:09.385978 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fbde1-3a41-464e-ac07-9d31845a08bc None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:09.386481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7fbde1-3a41-464e-ac07-9d31845a08bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:09.408214 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0a7fbde1-3a41-464e-ac07-9d31845a08bc None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:09.409845 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0a7fbde1-3a41-464e-ac07-9d31845a08bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:09.482281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 150/296] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:09.494181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b070607-7bcb-4fa9-9d4a-9edc2e3b4318 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:09.494464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b070607-7bcb-4fa9-9d4a-9edc2e3b4318 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:09.494739 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b070607-7bcb-4fa9-9d4a-9edc2e3b4318 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:09.497015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 147/297] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:09.502425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db4a1c-6d19-416d-b4fe-6751fba48f32 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:09.502707 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db4a1c-6d19-416d-b4fe-6751fba48f32 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:09.502978 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79db4a1c-6d19-416d-b4fe-6751fba48f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:09.522296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-79db4a1c-6d19-416d-b4fe-6751fba48f32 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:09.523491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-79db4a1c-6d19-416d-b4fe-6751fba48f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:09.596025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 151/298] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:59:09.784385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf317cd-907d-416b-9b4b-b4e48866895c None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:09.784910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf317cd-907d-416b-9b4b-b4e48866895c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:09.784910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf317cd-907d-416b-9b4b-b4e48866895c None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:09.786588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 148/299] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:09.793232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db4a1c-6d19-416d-b4fe-6751fba48f32 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:09.795050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905568b-4651-4972-b6a8-0fe3a9c739b5 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:09.795435 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905568b-4651-4972-b6a8-0fe3a9c739b5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:09.795927 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905568b-4651-4972-b6a8-0fe3a9c739b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:09.814745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4905568b-4651-4972-b6a8-0fe3a9c739b5 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:09.815921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4905568b-4651-4972-b6a8-0fe3a9c739b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:09.850365 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:09.898489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 149/300] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:59:09.899326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905568b-4651-4972-b6a8-0fe3a9c739b5 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:09.923715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79db4a1c-6d19-416d-b4fe-6751fba48f32 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:09.944972 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:09.975327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ca3a7-6e82-425f-a426-49ddf46dcdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:09.975959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ca3a7-6e82-425f-a426-49ddf46dcdd7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:09.976232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ca3a7-6e82-425f-a426-49ddf46dcdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:09.976496 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5ca3a7-6e82-425f-a426-49ddf46dcdd7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:09.989951 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5ca3a7-6e82-425f-a426-49ddf46dcdd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:09.995210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-7f5ca3a7-6e82-425f-a426-49ddf46dcdd7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:59:09.995210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:09.995210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:09.995210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:09.995210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:09.995210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:09.995210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:09.995210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:09.995210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:09.995210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:09.995210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:09.995210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:09.995210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:09.995210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:09.995210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:09.995210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:09.996307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:09.996307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:09.996307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:09.996307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:09.996307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:09.996307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:09.996307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:09.996307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:09.996307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:09.996307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:09.996307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:09.996307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:09.996307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:09.996307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:09.996307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:09.996307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:09.997177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:09.997177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:09.997177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:09.997177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:09.997177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:09.997177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:09.997177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:09.997177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:09.997177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:09.997177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:09.997177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:59:09.997177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:09.997177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 152/301] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:10.006109 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ca3a7-6e82-425f-a426-49ddf46dcdd7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:10.027311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905568b-4651-4972-b6a8-0fe3a9c739b5 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:10.064938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d469e8-f365-4a18-a9de-3b64e8380323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:10.065822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d469e8-f365-4a18-a9de-3b64e8380323 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:10.066250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d469e8-f365-4a18-a9de-3b64e8380323 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:10.066680 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d469e8-f365-4a18-a9de-3b64e8380323 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:10.067954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d469e8-f365-4a18-a9de-3b64e8380323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:10.069599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d469e8-f365-4a18-a9de-3b64e8380323 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:10.073001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 150/302] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:10.078644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d469e8-f365-4a18-a9de-3b64e8380323 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:10.133175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5ca3a7-6e82-425f-a426-49ddf46dcdd7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:10.134824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:10.179075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9b9eb4-a9d4-45b3-9a6a-5ab79c42e146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:10.179675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b9eb4-a9d4-45b3-9a6a-5ab79c42e146 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:10.179946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b9eb4-a9d4-45b3-9a6a-5ab79c42e146 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:10.180211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9b9eb4-a9d4-45b3-9a6a-5ab79c42e146 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:10.180862 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b9eb4-a9d4-45b3-9a6a-5ab79c42e146 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:10.181153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b9eb4-a9d4-45b3-9a6a-5ab79c42e146 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:10.182526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9b9eb4-a9d4-45b3-9a6a-5ab79c42e146 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:10.190854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 153/303] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:10.192310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf40ec-46a2-4f48-9381-c6ba948d9af5 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:10.192602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf40ec-46a2-4f48-9381-c6ba948d9af5 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:10.192876 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf40ec-46a2-4f48-9381-c6ba948d9af5 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:10.194128 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 154/304] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:10.194824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf40ec-46a2-4f48-9381-c6ba948d9af5 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:10.207402 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d469e8-f365-4a18-a9de-3b64e8380323 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:10.217737 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:10.242059 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fba35d-a3bd-41e1-8daa-2fd6c541e10d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:10.242746 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fba35d-a3bd-41e1-8daa-2fd6c541e10d None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:10.243079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fba35d-a3bd-41e1-8daa-2fd6c541e10d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:10.243416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fba35d-a3bd-41e1-8daa-2fd6c541e10d None admin] PATH_INFO: `/v3/groups` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:10.244099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fba35d-a3bd-41e1-8daa-2fd6c541e10d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:10.245651 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fba35d-a3bd-41e1-8daa-2fd6c541e10d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:10.260975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 151/305] 10.222.0.65 () {60 vars in 1134 bytes} [Wed Aug 31 18:59:10 2022] POST /identity/v3/groups => generated 229 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 18:59:10.260975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fba35d-a3bd-41e1-8daa-2fd6c541e10d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:10.300815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf40ec-46a2-4f48-9381-c6ba948d9af5 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:10.342909 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdbfef4-a08e-41cc-82aa-36756098ae61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:10.343629 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbfef4-a08e-41cc-82aa-36756098ae61 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:10.343905 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbfef4-a08e-41cc-82aa-36756098ae61 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:10.344248 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdbfef4-a08e-41cc-82aa-36756098ae61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:10.358812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdbfef4-a08e-41cc-82aa-36756098ae61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:10.364530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-6cdbfef4-a08e-41cc-82aa-36756098ae61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:10.364530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:10.364530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:10.364530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:10.364530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:10.364530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:10.364530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:10.364530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:10.364530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:10.364530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:10.364530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:10.364530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:10.364530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:10.364530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:10.364530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:10.364530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:10.365720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:10.365720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:10.365720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:10.365720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:10.365720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:10.365720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:10.365720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:10.365720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:10.365720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:10.365720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:10.365720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:10.365720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:10.365720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:10.365720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:10.365720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:10.365720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:10.370013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:10.370013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:10.370013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:10.370013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:10.370013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:10.370013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:10.370013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:10.370013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:10.370013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:10.370013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:10.370013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:10.370013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:10.370013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 155/306] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:10.370013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a409d6-6df8-49bf-a887-05554ec80d04 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:10.370774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a409d6-6df8-49bf-a887-05554ec80d04 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:10.370774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a409d6-6df8-49bf-a887-05554ec80d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:10.388308 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f2a409d6-6df8-49bf-a887-05554ec80d04 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:10.389615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f2a409d6-6df8-49bf-a887-05554ec80d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:10.389875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fba35d-a3bd-41e1-8daa-2fd6c541e10d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:10.413403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:10.432064 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a4ea0-69ce-4732-988c-2f69049d90af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:10.432996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a4ea0-69ce-4732-988c-2f69049d90af None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:10.432996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a4ea0-69ce-4732-988c-2f69049d90af None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:10.433305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0a4ea0-69ce-4732-988c-2f69049d90af None admin] PATH_INFO: `/v3/users/f2d17980e771476a862bac8ea67be92a` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:10.449260 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a4ea0-69ce-4732-988c-2f69049d90af None admin] RBAC: Authorizing `identity:get_user(user_id=f2d17980e771476a862bac8ea67be92a)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:10.450543 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0a4ea0-69ce-4732-988c-2f69049d90af None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:10.452390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 152/307] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:59:10 2022] GET /identity/v3/users/f2d17980e771476a862bac8ea67be92a => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:10.453069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a4ea0-69ce-4732-988c-2f69049d90af None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:10.480298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 156/308] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:10.481172 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a409d6-6df8-49bf-a887-05554ec80d04 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:10.486720 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:10.594491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a409d6-6df8-49bf-a887-05554ec80d04 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:10.598643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0a4ea0-69ce-4732-988c-2f69049d90af None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:10.630695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67c00b-9d6a-47a8-b8a5-6404bd55bfdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:10.631319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67c00b-9d6a-47a8-b8a5-6404bd55bfdf None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:10.631615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67c00b-9d6a-47a8-b8a5-6404bd55bfdf None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:10.632152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f67c00b-9d6a-47a8-b8a5-6404bd55bfdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:10.632658 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67c00b-9d6a-47a8-b8a5-6404bd55bfdf None admin] RBAC: Adding query filter params (role.id=2d35524d275d4bc4b09ce775ddec4963, user.id=f2d17980e771476a862bac8ea67be92a, scope.system=all) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:10.632958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67c00b-9d6a-47a8-b8a5-6404bd55bfdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:10.634460 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f67c00b-9d6a-47a8-b8a5-6404bd55bfdf None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:10.640991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 157/309] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Aug 31 18:59:10 2022] GET /identity/v3/role_assignments?user.id=f2d17980e771476a862bac8ea67be92a&scope.system=all&role.id=2d35524d275d4bc4b09ce775ddec4963 => generated 226 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:10.645439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58626d-32ce-4c57-8aca-377e72ed468f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:10.646387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58626d-32ce-4c57-8aca-377e72ed468f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:10.646867 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58626d-32ce-4c57-8aca-377e72ed468f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:10.647343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd58626d-32ce-4c57-8aca-377e72ed468f None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:10.648282 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58626d-32ce-4c57-8aca-377e72ed468f None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:10.648809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58626d-32ce-4c57-8aca-377e72ed468f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:10.650458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd58626d-32ce-4c57-8aca-377e72ed468f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:10.660518 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 153/310] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:10.666918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67c00b-9d6a-47a8-b8a5-6404bd55bfdf None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:10.689264 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:10.757762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58626d-32ce-4c57-8aca-377e72ed468f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:10.780790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f67c00b-9d6a-47a8-b8a5-6404bd55bfdf None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:10.825760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2355b-2d45-4b52-afd6-79b3904ae583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:10.826811 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2355b-2d45-4b52-afd6-79b3904ae583 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:10.827392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2355b-2d45-4b52-afd6-79b3904ae583 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:10.827871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc2355b-2d45-4b52-afd6-79b3904ae583 None admin] PATH_INFO: `/v3/users/45f5041aca054b99ad52b126be30ee1b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:10.829926 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2355b-2d45-4b52-afd6-79b3904ae583 None admin] RBAC: Authorizing `identity:get_user(user_id=45f5041aca054b99ad52b126be30ee1b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:10.832143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc2355b-2d45-4b52-afd6-79b3904ae583 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:10.839609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 158/311] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:59:10 2022] GET /identity/v3/users/45f5041aca054b99ad52b126be30ee1b => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:10.841690 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2355b-2d45-4b52-afd6-79b3904ae583 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:10.868171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:10.885161 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd58626d-32ce-4c57-8aca-377e72ed468f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:10.923365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe58dd-5ac4-4c7a-a994-f0bbc576f3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:10.924410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe58dd-5ac4-4c7a-a994-f0bbc576f3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:10.924687 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe58dd-5ac4-4c7a-a994-f0bbc576f3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:10.924955 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe58dd-5ac4-4c7a-a994-f0bbc576f3a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:10.937967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe58dd-5ac4-4c7a-a994-f0bbc576f3a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:10.944131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-c6fe58dd-5ac4-4c7a-a994-f0bbc576f3a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:59:10.944131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:10.944131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:10.944131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:10.944131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:10.944131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:10.944131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:10.944131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:10.944131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:10.944131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:10.944131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:10.944131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:10.944131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:10.944131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:10.944131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:10.944131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:10.945039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:10.945039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:10.945039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:10.945039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:10.945039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:10.945039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:10.945039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:10.945039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:10.945039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:10.945039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:10.945039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:10.945039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:10.945039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:10.945039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:10.945039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:10.945039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:10.946476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:10.946476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:10.946476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:10.946476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:10.946476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:10.946476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:10.946476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:10.946476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:10.946476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:10.946476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:10.946476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:59:10.946476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:10.946476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 154/312] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 18:59:10.951038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe58dd-5ac4-4c7a-a994-f0bbc576f3a5 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:10.966165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:10.989084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc2355b-2d45-4b52-afd6-79b3904ae583 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:11.033340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dc224-0d5f-4db5-b425-07ffb17e1d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:11.034433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dc224-0d5f-4db5-b425-07ffb17e1d7b None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:11.034720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dc224-0d5f-4db5-b425-07ffb17e1d7b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:11.035113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dc224-0d5f-4db5-b425-07ffb17e1d7b None admin] PATH_INFO: `/v3/system/users/45f5041aca054b99ad52b126be30ee1b/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:11.037123 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220dc224-0d5f-4db5-b425-07ffb17e1d7b None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=45f5041aca054b99ad52b126be30ee1b, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:11.038613 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220dc224-0d5f-4db5-b425-07ffb17e1d7b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:11.049561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 159/313] 10.222.0.65 () {58 vars in 1334 bytes} [Wed Aug 31 18:59:10 2022] PUT /identity/v3/system/users/45f5041aca054b99ad52b126be30ee1b/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 18:59:11.071910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe58dd-5ac4-4c7a-a994-f0bbc576f3a5 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:11.110468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26abaf5-33c1-47f6-b657-9690001ef7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:11.111342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26abaf5-33c1-47f6-b657-9690001ef7b5 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:11.111765 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26abaf5-33c1-47f6-b657-9690001ef7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:11.112178 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26abaf5-33c1-47f6-b657-9690001ef7b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:11.113025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26abaf5-33c1-47f6-b657-9690001ef7b5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:11.113514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26abaf5-33c1-47f6-b657-9690001ef7b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:11.115066 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26abaf5-33c1-47f6-b657-9690001ef7b5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:11.127808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 155/314] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:10 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:11.134859 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dc224-0d5f-4db5-b425-07ffb17e1d7b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:11.252828 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dc224-0d5f-4db5-b425-07ffb17e1d7b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:11.288921 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3bcb3-37b4-49a2-98cb-9a975e0c5c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:11.289985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3bcb3-37b4-49a2-98cb-9a975e0c5c29 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:11.290460 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3bcb3-37b4-49a2-98cb-9a975e0c5c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:11.290928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda3bcb3-37b4-49a2-98cb-9a975e0c5c29 None admin] PATH_INFO: `/v3/users/8288f9bc86f244a3879ef7de7e3b7625` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:11.292311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3bcb3-37b4-49a2-98cb-9a975e0c5c29 None admin] RBAC: Authorizing `identity:get_user(user_id=8288f9bc86f244a3879ef7de7e3b7625)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:11.294601 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda3bcb3-37b4-49a2-98cb-9a975e0c5c29 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:11.296772 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 160/315] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:59:11 2022] GET /identity/v3/users/8288f9bc86f244a3879ef7de7e3b7625 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:11.302232 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26abaf5-33c1-47f6-b657-9690001ef7b5 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:11.325664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:11.409853 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26abaf5-33c1-47f6-b657-9690001ef7b5 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:11.450498 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150349e-3749-412c-84e1-9e7371d7574f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:11.451114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150349e-3749-412c-84e1-9e7371d7574f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:11.451395 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150349e-3749-412c-84e1-9e7371d7574f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:11.451670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150349e-3749-412c-84e1-9e7371d7574f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:11.521722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150349e-3749-412c-84e1-9e7371d7574f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:11.523190 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150349e-3749-412c-84e1-9e7371d7574f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:11.538255 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:11.633849 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 156/316] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:11.640240 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3bcb3-37b4-49a2-98cb-9a975e0c5c29 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:11.661460 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:11.750439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda3bcb3-37b4-49a2-98cb-9a975e0c5c29 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:11.786818 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a4f63-189d-40c4-bddd-561d8e58f756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:11.787451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a4f63-189d-40c4-bddd-561d8e58f756 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:11.787729 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a4f63-189d-40c4-bddd-561d8e58f756 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:11.788003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a4f63-189d-40c4-bddd-561d8e58f756 None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:11.789236 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a4f63-189d-40c4-bddd-561d8e58f756 None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:11.790678 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a4f63-189d-40c4-bddd-561d8e58f756 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:11.792701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 161/317] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:11 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:11.798438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150349e-3749-412c-84e1-9e7371d7574f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:11.904477 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150349e-3749-412c-84e1-9e7371d7574f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:11.915287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:11.939086 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebc4c52-c245-4e9a-893e-9cf1ef83e3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:11.939682 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc4c52-c245-4e9a-893e-9cf1ef83e3ba None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:11.939955 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc4c52-c245-4e9a-893e-9cf1ef83e3ba None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:11.940222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebc4c52-c245-4e9a-893e-9cf1ef83e3ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:11.940935 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc4c52-c245-4e9a-893e-9cf1ef83e3ba None admin] RBAC: Adding query filter params (role.id=2d35524d275d4bc4b09ce775ddec4963, scope.project.id=8ab3b89f1290419e93f22ea7eee77da8, user.id=8288f9bc86f244a3879ef7de7e3b7625) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:11.941258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc4c52-c245-4e9a-893e-9cf1ef83e3ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:11.942659 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebc4c52-c245-4e9a-893e-9cf1ef83e3ba None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:11.949243 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 157/318] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:59:11 2022] GET /identity/v3/role_assignments?user.id=8288f9bc86f244a3879ef7de7e3b7625&scope.project.id=8ab3b89f1290419e93f22ea7eee77da8&role.id=2d35524d275d4bc4b09ce775ddec4963 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:12.290830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8db8d6-2f1a-41d5-93cd-2e9eb1c6f39f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:12.291361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8db8d6-2f1a-41d5-93cd-2e9eb1c6f39f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:12.291361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8db8d6-2f1a-41d5-93cd-2e9eb1c6f39f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:12.292893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 162/319] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:12.297761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9a31f-11bf-480d-8a00-aeb552a6ba4e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:12.298044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9a31f-11bf-480d-8a00-aeb552a6ba4e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:12.298313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad9a31f-11bf-480d-8a00-aeb552a6ba4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:12.316786 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-1ad9a31f-11bf-480d-8a00-aeb552a6ba4e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:12.318014 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1ad9a31f-11bf-480d-8a00-aeb552a6ba4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:12.387082 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 158/320] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:12.606240 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe815d-31df-47f4-a381-fcaa835c3e6d None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:12.606631 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe815d-31df-47f4-a381-fcaa835c3e6d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:12.606802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe815d-31df-47f4-a381-fcaa835c3e6d None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:12.608274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 163/321] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:12.613205 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59d7c5-7a49-4980-a6ef-9815cfbf157a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:12.613494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59d7c5-7a49-4980-a6ef-9815cfbf157a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:12.613771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e59d7c5-7a49-4980-a6ef-9815cfbf157a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:12.632563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-1e59d7c5-7a49-4980-a6ef-9815cfbf157a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:12.633774 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1e59d7c5-7a49-4980-a6ef-9815cfbf157a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:12.680404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe815d-31df-47f4-a381-fcaa835c3e6d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:12.708507 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 159/322] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:12.771255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:12.788321 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe815d-31df-47f4-a381-fcaa835c3e6d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:12.821591 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf524677-cc5c-4fe7-bdf2-f0c293ce1ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:12.822215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf524677-cc5c-4fe7-bdf2-f0c293ce1ada None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:12.822494 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf524677-cc5c-4fe7-bdf2-f0c293ce1ada None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:12.822764 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf524677-cc5c-4fe7-bdf2-f0c293ce1ada None admin] PATH_INFO: `/v3/domains/default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:12.823894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf524677-cc5c-4fe7-bdf2-f0c293ce1ada None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:12.825349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf524677-cc5c-4fe7-bdf2-f0c293ce1ada None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:12.826919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 164/323] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:12.832463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59d7c5-7a49-4980-a6ef-9815cfbf157a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:12.867631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:12.943425 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e59d7c5-7a49-4980-a6ef-9815cfbf157a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:12.978675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf524677-cc5c-4fe7-bdf2-f0c293ce1ada None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:12.987673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21ad371-47a0-4128-bcb6-6f70573b9104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:12.988639 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ad371-47a0-4128-bcb6-6f70573b9104 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:12.989775 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ad371-47a0-4128-bcb6-6f70573b9104 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:12.990184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21ad371-47a0-4128-bcb6-6f70573b9104 None admin] PATH_INFO: `/v3/groups` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:12.991745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ad371-47a0-4128-bcb6-6f70573b9104 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:12.994755 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21ad371-47a0-4128-bcb6-6f70573b9104 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:13.017378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 160/324] 10.222.0.65 () {60 vars in 1134 bytes} [Wed Aug 31 18:59:12 2022] POST /identity/v3/groups => generated 226 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:59:13.088509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bc5d5-e9cd-454e-bdcc-b15fad6b1a3f None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:13.088851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bc5d5-e9cd-454e-bdcc-b15fad6b1a3f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:13.089053 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bc5d5-e9cd-454e-bdcc-b15fad6b1a3f None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:13.091217 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 161/325] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 18:59:13.096658 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a4e0d-b573-4409-87a5-7a0ac310e8c3 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:13.096939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a4e0d-b573-4409-87a5-7a0ac310e8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:13.097234 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a4e0d-b573-4409-87a5-7a0ac310e8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:13.106630 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf524677-cc5c-4fe7-bdf2-f0c293ce1ada None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:13.117477 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-de1a4e0d-b573-4409-87a5-7a0ac310e8c3 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:13.118984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-de1a4e0d-b573-4409-87a5-7a0ac310e8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:13.143139 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c0079-5421-4447-87a9-8637ace3618e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:13.143785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c0079-5421-4447-87a9-8637ace3618e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:13.144063 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c0079-5421-4447-87a9-8637ace3618e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:13.144357 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010c0079-5421-4447-87a9-8637ace3618e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:13.155987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010c0079-5421-4447-87a9-8637ace3618e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:13.161338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-010c0079-5421-4447-87a9-8637ace3618e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:59:13.161338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:13.161338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:13.161338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:13.161338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:13.161338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:13.161338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:13.161338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:13.161338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:13.161338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:13.161338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:13.161338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:13.161338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:13.161338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:13.161338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:13.161338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:13.162210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:13.162210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:13.162210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:13.162210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:13.162210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:13.162210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:13.162210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:13.162210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:13.162210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:13.162210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:13.162210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:13.162210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:13.162210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:13.162210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:13.162210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:13.162210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:13.162977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:13.162977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:13.162977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:13.162977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:13.162977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:13.162977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:13.162977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:13.162977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:13.162977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:13.162977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:13.162977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:59:13.162977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:13.162977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 165/326] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 18:59:13.168521 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c0079-5421-4447-87a9-8637ace3618e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:13.186514 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:13.202555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 162/327] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:59:13.286495 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010c0079-5421-4447-87a9-8637ace3618e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:13.327908 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb992c71-31c7-452f-9bd2-ee25d0d9498d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:13.328546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb992c71-31c7-452f-9bd2-ee25d0d9498d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:13.328800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb992c71-31c7-452f-9bd2-ee25d0d9498d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:13.329161 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb992c71-31c7-452f-9bd2-ee25d0d9498d None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:13.329857 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb992c71-31c7-452f-9bd2-ee25d0d9498d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:13.330162 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb992c71-31c7-452f-9bd2-ee25d0d9498d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:13.331574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb992c71-31c7-452f-9bd2-ee25d0d9498d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:13.341610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 166/328] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:13.347327 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a4e0d-b573-4409-87a5-7a0ac310e8c3 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:13.376235 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:13.460925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a4e0d-b573-4409-87a5-7a0ac310e8c3 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:13.491316 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb992c71-31c7-452f-9bd2-ee25d0d9498d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:13.501470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2247c15-320a-4f95-a7ed-fb70374a15a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:13.502108 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2247c15-320a-4f95-a7ed-fb70374a15a1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:13.502388 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2247c15-320a-4f95-a7ed-fb70374a15a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:13.502661 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2247c15-320a-4f95-a7ed-fb70374a15a1 None admin] PATH_INFO: `/v3/users/f2d17980e771476a862bac8ea67be92a` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:13.504560 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2247c15-320a-4f95-a7ed-fb70374a15a1 None admin] RBAC: Authorizing `identity:get_user(user_id=f2d17980e771476a862bac8ea67be92a)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:13.506120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2247c15-320a-4f95-a7ed-fb70374a15a1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:13.507710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 163/329] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:59:13 2022] GET /identity/v3/users/f2d17980e771476a862bac8ea67be92a => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:13.515559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2247c15-320a-4f95-a7ed-fb70374a15a1 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:13.591868 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:13.614304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb992c71-31c7-452f-9bd2-ee25d0d9498d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:13.615528 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:13.628622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2247c15-320a-4f95-a7ed-fb70374a15a1 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:13.652321 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c5333-7015-4bc2-855b-1704e874db7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:13.653057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c5333-7015-4bc2-855b-1704e874db7c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:13.653472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c5333-7015-4bc2-855b-1704e874db7c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:13.653823 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8c5333-7015-4bc2-855b-1704e874db7c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:13.667217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8c5333-7015-4bc2-855b-1704e874db7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:13.672931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242f3a1-3be5-47a6-bad1-08fde1db988a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:13.673835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242f3a1-3be5-47a6-bad1-08fde1db988a None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:13.674098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-6b8c5333-7015-4bc2-855b-1704e874db7c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:13.674098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:13.674098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:13.674098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:13.674098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:13.674098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:13.674098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:13.674098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:13.674098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:13.674098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:13.674098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:13.674098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:13.674098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:13.674098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:13.674098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:13.674098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:13.675014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:13.675014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:13.675014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:13.675014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:13.675014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:13.675014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:13.675014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:13.675014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:13.675014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:13.675014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:13.675014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:13.675014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:13.675014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:13.675014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:13.675014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:13.675014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:13.675830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:13.675830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:13.675830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:13.675830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:13.675830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:13.675830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:13.675830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:13.675830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:13.675830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:13.675830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:13.675830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:13.675830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:13.676423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242f3a1-3be5-47a6-bad1-08fde1db988a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:13.676423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2242f3a1-3be5-47a6-bad1-08fde1db988a None admin] PATH_INFO: `/v3/system/users/f2d17980e771476a862bac8ea67be92a/roles/2d35524d275d4bc4b09ce775ddec4963` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:13.676548 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 167/330] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 18:59:13.678612 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242f3a1-3be5-47a6-bad1-08fde1db988a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f2d17980e771476a862bac8ea67be92a, role_id=2d35524d275d4bc4b09ce775ddec4963)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:13.678910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2242f3a1-3be5-47a6-bad1-08fde1db988a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:13.680920 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c5333-7015-4bc2-855b-1704e874db7c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:13.693102 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 164/331] 10.222.0.65 () {58 vars in 1334 bytes} [Wed Aug 31 18:59:13 2022] PUT /identity/v3/system/users/f2d17980e771476a862bac8ea67be92a/roles/2d35524d275d4bc4b09ce775ddec4963 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 18:59:13.804344 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8c5333-7015-4bc2-855b-1704e874db7c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:13.844398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff780ebd-a43c-4406-aec6-dbf0244f233e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:13.845079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff780ebd-a43c-4406-aec6-dbf0244f233e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:13.845393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff780ebd-a43c-4406-aec6-dbf0244f233e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:13.845760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff780ebd-a43c-4406-aec6-dbf0244f233e None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:13.846435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff780ebd-a43c-4406-aec6-dbf0244f233e None admin] RBAC: Adding query filter params (name=member) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:13.846735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff780ebd-a43c-4406-aec6-dbf0244f233e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:13.848143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff780ebd-a43c-4406-aec6-dbf0244f233e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:13.850901 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:13.859806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 168/332] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:13.865328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242f3a1-3be5-47a6-bad1-08fde1db988a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:13.980855 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2242f3a1-3be5-47a6-bad1-08fde1db988a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:14.020765 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6816c7e1-c0b2-4db3-a0a1-01dc2205e4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:14.021433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816c7e1-c0b2-4db3-a0a1-01dc2205e4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:14.021714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816c7e1-c0b2-4db3-a0a1-01dc2205e4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:14.021988 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6816c7e1-c0b2-4db3-a0a1-01dc2205e4f5 None admin] PATH_INFO: `/v3/users/45f5041aca054b99ad52b126be30ee1b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:14.023126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816c7e1-c0b2-4db3-a0a1-01dc2205e4f5 None admin] RBAC: Authorizing `identity:get_user(user_id=45f5041aca054b99ad52b126be30ee1b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:14.024552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6816c7e1-c0b2-4db3-a0a1-01dc2205e4f5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:14.026478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 165/333] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:59:13 2022] GET /identity/v3/users/45f5041aca054b99ad52b126be30ee1b => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:14.032535 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff780ebd-a43c-4406-aec6-dbf0244f233e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:14.139480 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff780ebd-a43c-4406-aec6-dbf0244f233e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:14.175030 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03cfa73-abef-4451-af1e-0f81fbe66ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:14.175690 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cfa73-abef-4451-af1e-0f81fbe66ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:14.176008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cfa73-abef-4451-af1e-0f81fbe66ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:14.176275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03cfa73-abef-4451-af1e-0f81fbe66ba7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:14.177009 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cfa73-abef-4451-af1e-0f81fbe66ba7 None admin] RBAC: Adding query filter params (role.id=b80a0d1a8d164b53bef5726b92e68676, user.id=45f5041aca054b99ad52b126be30ee1b, scope.system=all) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:14.177455 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cfa73-abef-4451-af1e-0f81fbe66ba7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:14.178784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03cfa73-abef-4451-af1e-0f81fbe66ba7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:14.192381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 169/334] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Aug 31 18:59:14 2022] GET /identity/v3/role_assignments?user.id=45f5041aca054b99ad52b126be30ee1b&scope.system=all&role.id=b80a0d1a8d164b53bef5726b92e68676 => generated 510 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:15.332576 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80843686-51c8-44eb-8061-8d509dcde4a6 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:15.333132 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80843686-51c8-44eb-8061-8d509dcde4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:15.333132 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80843686-51c8-44eb-8061-8d509dcde4a6 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:15.334701 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 166/335] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:15.341993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c67532-4d08-492f-949b-ab7371e8c615 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:15.342545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c67532-4d08-492f-949b-ab7371e8c615 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:15.342545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c67532-4d08-492f-949b-ab7371e8c615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:15.364196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c1c67532-4d08-492f-949b-ab7371e8c615 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:15.365413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c1c67532-4d08-492f-949b-ab7371e8c615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:15.426183 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:15.447735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 170/336] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:15.470629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6fc76-4868-449b-b7a7-33b59e68c42c None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:15.470945 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6fc76-4868-449b-b7a7-33b59e68c42c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:15.471201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e6fc76-4868-449b-b7a7-33b59e68c42c None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:15.472801 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 167/337] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:15.490823 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff26e4-9218-4130-9ce3-315dc9fb7ef8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:15.491126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff26e4-9218-4130-9ce3-315dc9fb7ef8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:15.491351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bff26e4-9218-4130-9ce3-315dc9fb7ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:15.509783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0bff26e4-9218-4130-9ce3-315dc9fb7ef8 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:15.510952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0bff26e4-9218-4130-9ce3-315dc9fb7ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:15.599774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 171/338] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:59:15.999997 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6fc76-4868-449b-b7a7-33b59e68c42c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:16.032696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:16.129908 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e6fc76-4868-449b-b7a7-33b59e68c42c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:16.148378 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff26e4-9218-4130-9ce3-315dc9fb7ef8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:16.188634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a042d1e-981d-4511-b96c-0befa4aa49a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:16.188634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a042d1e-981d-4511-b96c-0befa4aa49a1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:16.188928 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a042d1e-981d-4511-b96c-0befa4aa49a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:16.189152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a042d1e-981d-4511-b96c-0befa4aa49a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:16.215312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a042d1e-981d-4511-b96c-0befa4aa49a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:16.221819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-0a042d1e-981d-4511-b96c-0befa4aa49a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:16.221819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:16.221819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:16.221819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:16.221819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:16.221819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:16.221819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:16.221819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:16.221819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:16.221819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:16.221819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:16.221819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:16.221819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:16.221819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:16.221819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:16.221819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:16.223001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:16.223001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:16.223001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:16.223001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:16.223001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:16.223001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:16.223001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:16.223001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:16.223001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:16.223001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:16.223001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:16.223001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:16.223001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:16.223001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:16.223001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:16.223001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:16.223802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:16.223802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:16.223802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:16.223802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:16.223802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:16.223802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:16.223802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:16.223802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:16.223802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:16.223802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:16.223802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:16.223802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:16.226311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 168/339] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:16.231456 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a042d1e-981d-4511-b96c-0befa4aa49a1 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:16.327563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bff26e4-9218-4130-9ce3-315dc9fb7ef8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:16.369804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:16.403940 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693a8e3-1fa1-469b-ac5b-03a3c64c323e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:16.404630 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693a8e3-1fa1-469b-ac5b-03a3c64c323e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:16.404912 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693a8e3-1fa1-469b-ac5b-03a3c64c323e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:16.405201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693a8e3-1fa1-469b-ac5b-03a3c64c323e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:16.425124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:16.429012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693a8e3-1fa1-469b-ac5b-03a3c64c323e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:16.435057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-0693a8e3-1fa1-469b-ac5b-03a3c64c323e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:59:16.435057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:16.435057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:16.435057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:16.435057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:16.435057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:16.435057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:16.435057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:16.435057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:16.435057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:16.435057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:16.435057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:16.435057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:16.435057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:16.435057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:16.435057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:16.435939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:16.435939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:16.435939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:16.435939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:16.435939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:16.435939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:16.435939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:16.435939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:16.435939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:16.435939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:16.435939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:16.435939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:16.435939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:16.435939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:16.435939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:16.435939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:16.436756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:16.436756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:16.436756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:16.436756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:16.436756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:16.436756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:16.436756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:16.436756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:16.436756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:16.436756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:16.436756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:59:16.436756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:16.441219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 172/340] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 18:59:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:16.441492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2452049-20b0-4882-82bd-1fdbed4b020d None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:16.441781 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2452049-20b0-4882-82bd-1fdbed4b020d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:16.442498 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2452049-20b0-4882-82bd-1fdbed4b020d None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:16.443988 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 173/341] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:16.448685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2452049-20b0-4882-82bd-1fdbed4b020d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:16.465413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a042d1e-981d-4511-b96c-0befa4aa49a1 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:16.513901 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0201e-ccf7-4763-8743-e021431fb67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:16.514895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0201e-ccf7-4763-8743-e021431fb67b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:16.515389 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0201e-ccf7-4763-8743-e021431fb67b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:16.515912 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0201e-ccf7-4763-8743-e021431fb67b None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:16.516774 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0201e-ccf7-4763-8743-e021431fb67b None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:16.517389 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0201e-ccf7-4763-8743-e021431fb67b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:16.519234 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0201e-ccf7-4763-8743-e021431fb67b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:16.531025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 169/342] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:16.532807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a85f7-9c0d-4fd6-8c55-83af8f966687 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:16.533446 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a85f7-9c0d-4fd6-8c55-83af8f966687 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:16.533924 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a85f7-9c0d-4fd6-8c55-83af8f966687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:16.566522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-429a85f7-9c0d-4fd6-8c55-83af8f966687 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:16.568886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-429a85f7-9c0d-4fd6-8c55-83af8f966687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:16.622811 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2452049-20b0-4882-82bd-1fdbed4b020d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:16.656716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 170/343] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:16.658302 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a85f7-9c0d-4fd6-8c55-83af8f966687 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:16.667079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fd2c8d-cd40-47e8-9460-d3555b00b5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:16.668520 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd2c8d-cd40-47e8-9460-d3555b00b5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:16.668942 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd2c8d-cd40-47e8-9460-d3555b00b5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:16.669296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fd2c8d-cd40-47e8-9460-d3555b00b5b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:16.670362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd2c8d-cd40-47e8-9460-d3555b00b5b7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:16.671144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd2c8d-cd40-47e8-9460-d3555b00b5b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:16.673464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fd2c8d-cd40-47e8-9460-d3555b00b5b7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:16.687753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 174/344] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:16.689771 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91541222-6626-49dc-8118-8247c48a2658 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:16.690245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91541222-6626-49dc-8118-8247c48a2658 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:16.690719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91541222-6626-49dc-8118-8247c48a2658 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:16.692813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 175/345] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:16.693959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91541222-6626-49dc-8118-8247c48a2658 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:16.757684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:16.768392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:16.822267 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a85f7-9c0d-4fd6-8c55-83af8f966687 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:16.872695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91541222-6626-49dc-8118-8247c48a2658 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:16.885986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4f6080-0cf6-4f4a-9a13-c583c33a0167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:16.887062 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f6080-0cf6-4f4a-9a13-c583c33a0167 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:16.887319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f6080-0cf6-4f4a-9a13-c583c33a0167 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:16.887580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4f6080-0cf6-4f4a-9a13-c583c33a0167 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:16.935910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b82f6-3ec8-43ee-a4b2-d26ed707ac78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:16.937282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b82f6-3ec8-43ee-a4b2-d26ed707ac78 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:16.937760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b82f6-3ec8-43ee-a4b2-d26ed707ac78 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:16.938068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8b82f6-3ec8-43ee-a4b2-d26ed707ac78 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:16.997739 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f6080-0cf6-4f4a-9a13-c583c33a0167 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:16.999436 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4f6080-0cf6-4f4a-9a13-c583c33a0167 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:17.035095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b82f6-3ec8-43ee-a4b2-d26ed707ac78 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:17.036651 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8b82f6-3ec8-43ee-a4b2-d26ed707ac78 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:17.080097 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:17.098918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:17.132306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 171/346] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:17.133676 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6787c05-4311-4fea-b194-064ea81aafce None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:17.133966 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6787c05-4311-4fea-b194-064ea81aafce None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:17.134234 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6787c05-4311-4fea-b194-064ea81aafce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:17.156711 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 176/347] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:17.156874 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b6787c05-4311-4fea-b194-064ea81aafce None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:17.157865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b6787c05-4311-4fea-b194-064ea81aafce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:17.158251 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b82f6-3ec8-43ee-a4b2-d26ed707ac78 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:17.247271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 172/348] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:17.248612 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6787c05-4311-4fea-b194-064ea81aafce None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:17.302074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8b82f6-3ec8-43ee-a4b2-d26ed707ac78 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:17.342704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca917d3c-7ed9-4ea8-834e-eb56e144d51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:17.343342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca917d3c-7ed9-4ea8-834e-eb56e144d51e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:17.343619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca917d3c-7ed9-4ea8-834e-eb56e144d51e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:17.343888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca917d3c-7ed9-4ea8-834e-eb56e144d51e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:17.351308 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:17.357005 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca917d3c-7ed9-4ea8-834e-eb56e144d51e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:17.362834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-ca917d3c-7ed9-4ea8-834e-eb56e144d51e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:17.362834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:17.362834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:17.362834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:17.362834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:17.362834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:17.362834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:17.362834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:17.362834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:17.362834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:17.362834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:17.362834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:17.362834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:17.362834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:17.362834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:17.362834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:17.363742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:17.363742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:17.363742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:17.363742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:17.363742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:17.363742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:17.363742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:17.363742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:17.363742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:17.363742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:17.363742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:17.363742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:17.363742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:17.363742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:17.363742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:17.363742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:17.364562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:17.364562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:17.364562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:17.364562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:17.364562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:17.364562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:17.364562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:17.364562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:17.364562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:17.364562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:17.364562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:17.364562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:17.367959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 177/349] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:17.369057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca917d3c-7ed9-4ea8-834e-eb56e144d51e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:17.372183 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6787c05-4311-4fea-b194-064ea81aafce None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:17.381628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:17.427313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e39365-041d-4630-a2d4-36103fe21554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:17.429879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e39365-041d-4630-a2d4-36103fe21554 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:17.430362 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e39365-041d-4630-a2d4-36103fe21554 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:17.430831 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e39365-041d-4630-a2d4-36103fe21554 None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:17.434867 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e39365-041d-4630-a2d4-36103fe21554 None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:17.437023 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e39365-041d-4630-a2d4-36103fe21554 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:17.440069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 173/350] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:17 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:17.442210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e39365-041d-4630-a2d4-36103fe21554 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:17.543234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca917d3c-7ed9-4ea8-834e-eb56e144d51e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:17.593490 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d13f2-63d2-4e7d-aaef-b57c8037996f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:17.594122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d13f2-63d2-4e7d-aaef-b57c8037996f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:17.594397 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d13f2-63d2-4e7d-aaef-b57c8037996f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:17.594666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55d13f2-63d2-4e7d-aaef-b57c8037996f None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:17.595793 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d13f2-63d2-4e7d-aaef-b57c8037996f None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:17.597262 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55d13f2-63d2-4e7d-aaef-b57c8037996f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:17.599020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 178/351] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:17 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:17.599944 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d13f2-63d2-4e7d-aaef-b57c8037996f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:17.619474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e39365-041d-4630-a2d4-36103fe21554 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:17.666873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f736da-e9c8-4e47-91ca-bdc17e3fd24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:17.668565 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f736da-e9c8-4e47-91ca-bdc17e3fd24e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:17.669151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f736da-e9c8-4e47-91ca-bdc17e3fd24e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:17.669691 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f736da-e9c8-4e47-91ca-bdc17e3fd24e None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:17.670549 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f736da-e9c8-4e47-91ca-bdc17e3fd24e None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:17.671046 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f736da-e9c8-4e47-91ca-bdc17e3fd24e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:17.672873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:17.673122 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f736da-e9c8-4e47-91ca-bdc17e3fd24e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:17.685468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 174/352] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:17.689807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f736da-e9c8-4e47-91ca-bdc17e3fd24e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:17.727012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55d13f2-63d2-4e7d-aaef-b57c8037996f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:17.739692 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:17.771434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1388e-db4a-4f0d-9347-2cc12755ad0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:17.773092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1388e-db4a-4f0d-9347-2cc12755ad0a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:17.773451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1388e-db4a-4f0d-9347-2cc12755ad0a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:17.773808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b1388e-db4a-4f0d-9347-2cc12755ad0a None admin] PATH_INFO: `/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:17.782272 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1388e-db4a-4f0d-9347-2cc12755ad0a None admin] RBAC: Authorizing `identity:get_group(group_id=9aa3e6fe034d493bb846ba2e61112d0b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:17.783733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b1388e-db4a-4f0d-9347-2cc12755ad0a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:17.785733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 179/353] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 18:59:17 2022] GET /identity/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:17.786778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1388e-db4a-4f0d-9347-2cc12755ad0a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:17.826140 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f736da-e9c8-4e47-91ca-bdc17e3fd24e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:17.878329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc412d-0eaa-4052-8457-039a8422e26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:17.880758 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc412d-0eaa-4052-8457-039a8422e26d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:17.880758 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc412d-0eaa-4052-8457-039a8422e26d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:17.880758 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cc412d-0eaa-4052-8457-039a8422e26d None admin] PATH_INFO: `/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:17.880974 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc412d-0eaa-4052-8457-039a8422e26d None admin] RBAC: Authorizing `identity:get_group(group_id=9aa3e6fe034d493bb846ba2e61112d0b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:17.882772 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cc412d-0eaa-4052-8457-039a8422e26d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:17.885136 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 175/354] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 18:59:17 2022] GET /identity/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:17.886058 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc412d-0eaa-4052-8457-039a8422e26d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:17.910479 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b1388e-db4a-4f0d-9347-2cc12755ad0a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:17.944959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:17.953870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3330b1a-38e7-403c-aad6-c3aaa2d27ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:17.953870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3330b1a-38e7-403c-aad6-c3aaa2d27ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:17.954111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3330b1a-38e7-403c-aad6-c3aaa2d27ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:17.954307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3330b1a-38e7-403c-aad6-c3aaa2d27ae6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:17.968070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3330b1a-38e7-403c-aad6-c3aaa2d27ae6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:17.973804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-e3330b1a-38e7-403c-aad6-c3aaa2d27ae6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:59:17.973804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:17.973804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:17.973804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:17.973804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:17.973804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:17.973804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:17.973804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:17.973804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:17.973804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:17.973804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:17.973804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:17.973804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:17.973804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:17.973804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:17.973804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:17.974688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:17.974688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:17.974688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:17.974688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:17.974688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:17.974688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:17.974688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:17.974688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:17.974688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:17.974688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:17.974688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:17.974688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:17.974688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:17.974688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:17.974688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:17.974688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:17.975511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:17.975511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:17.975511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:17.975511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:17.975511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:17.975511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:17.975511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:17.975511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:17.975511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:17.975511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:17.975511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 31 18:59:17.975511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:17.975511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 180/355] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:17.976167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75597218-9fb9-497a-996d-aee7fa07e39f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:17.976701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75597218-9fb9-497a-996d-aee7fa07e39f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:17.976701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75597218-9fb9-497a-996d-aee7fa07e39f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:17.981269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 181/356] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:17.981269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75597218-9fb9-497a-996d-aee7fa07e39f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:18.010393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cc412d-0eaa-4052-8457-039a8422e26d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:18.014099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:18.053720 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843739f-3e86-43c8-bfa1-1f64dc4ce412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:18.054339 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843739f-3e86-43c8-bfa1-1f64dc4ce412 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:18.054621 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843739f-3e86-43c8-bfa1-1f64dc4ce412 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:18.054896 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843739f-3e86-43c8-bfa1-1f64dc4ce412 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:18.103429 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75597218-9fb9-497a-996d-aee7fa07e39f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:18.136477 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843739f-3e86-43c8-bfa1-1f64dc4ce412 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:18.137945 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843739f-3e86-43c8-bfa1-1f64dc4ce412 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:18.146879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817c6bc-0f93-4d32-bb4d-d9ea2704045f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:18.147331 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817c6bc-0f93-4d32-bb4d-d9ea2704045f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:18.147607 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817c6bc-0f93-4d32-bb4d-d9ea2704045f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:18.147875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817c6bc-0f93-4d32-bb4d-d9ea2704045f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:18.149231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817c6bc-0f93-4d32-bb4d-d9ea2704045f None admin] RBAC: Adding query filter params (group.id=9aa3e6fe034d493bb846ba2e61112d0b, role.id=b80a0d1a8d164b53bef5726b92e68676, scope.project.id=7d21b4b8971a47ff8985309015eb53d4) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:18.149231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817c6bc-0f93-4d32-bb4d-d9ea2704045f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:18.150344 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817c6bc-0f93-4d32-bb4d-d9ea2704045f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:18.157161 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 182/357] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 18:59:17 2022] GET /identity/v3/role_assignments?group.id=9aa3e6fe034d493bb846ba2e61112d0b&scope.project.id=7d21b4b8971a47ff8985309015eb53d4&role.id=b80a0d1a8d164b53bef5726b92e68676 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:18.157932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817c6bc-0f93-4d32-bb4d-d9ea2704045f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:18.247783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:18.288044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 176/358] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:18.289520 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cc26a-ff35-4f4d-9dba-62faf5e037d2 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:18.289802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cc26a-ff35-4f4d-9dba-62faf5e037d2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:18.290074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37cc26a-ff35-4f4d-9dba-62faf5e037d2 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:18.291373 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 177/359] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:18.292062 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cc26a-ff35-4f4d-9dba-62faf5e037d2 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:18.297683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:18.313272 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817c6bc-0f93-4d32-bb4d-d9ea2704045f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:18.353382 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fea8b9b-2a94-471c-a9bb-efe8a8bc64da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:18.354327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea8b9b-2a94-471c-a9bb-efe8a8bc64da None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:18.354742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea8b9b-2a94-471c-a9bb-efe8a8bc64da None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:18.355292 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fea8b9b-2a94-471c-a9bb-efe8a8bc64da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:18.356059 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea8b9b-2a94-471c-a9bb-efe8a8bc64da None admin] RBAC: Adding query filter params (group.id=9aa3e6fe034d493bb846ba2e61112d0b, role.id=470b660efbe04083a803e79d8b647160, scope.project.id=7d21b4b8971a47ff8985309015eb53d4) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:18.356362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea8b9b-2a94-471c-a9bb-efe8a8bc64da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:18.358856 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fea8b9b-2a94-471c-a9bb-efe8a8bc64da None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:18.365343 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 183/360] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 18:59:18 2022] GET /identity/v3/role_assignments?group.id=9aa3e6fe034d493bb846ba2e61112d0b&scope.project.id=7d21b4b8971a47ff8985309015eb53d4&role.id=470b660efbe04083a803e79d8b647160 => generated 260 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:18.366592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6e023-adb2-4e77-8578-60e2bb66e672 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:18.366773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6e023-adb2-4e77-8578-60e2bb66e672 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:18.367038 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc6e023-adb2-4e77-8578-60e2bb66e672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:18.403444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-bbc6e023-adb2-4e77-8578-60e2bb66e672 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:18.405698 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bbc6e023-adb2-4e77-8578-60e2bb66e672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:18.425533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37cc26a-ff35-4f4d-9dba-62faf5e037d2 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:18.476602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfaea81-cfe2-4860-b8c8-277dc7a0c203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:18.477961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfaea81-cfe2-4860-b8c8-277dc7a0c203 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:18.478518 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfaea81-cfe2-4860-b8c8-277dc7a0c203 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:18.478954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfaea81-cfe2-4860-b8c8-277dc7a0c203 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:18.479770 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfaea81-cfe2-4860-b8c8-277dc7a0c203 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:18.480250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfaea81-cfe2-4860-b8c8-277dc7a0c203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:18.481853 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfaea81-cfe2-4860-b8c8-277dc7a0c203 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:18.492664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 178/361] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:18.493965 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfaea81-cfe2-4860-b8c8-277dc7a0c203 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:18.499709 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 184/362] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:18.500993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc886f-90e8-4471-9fbb-c8d96d905ae7 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:18.505503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc886f-90e8-4471-9fbb-c8d96d905ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:18.505978 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adc886f-90e8-4471-9fbb-c8d96d905ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:18.540917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-5adc886f-90e8-4471-9fbb-c8d96d905ae7 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:18.542283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5adc886f-90e8-4471-9fbb-c8d96d905ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:18.568067 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:18.597690 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:18.617147 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfaea81-cfe2-4860-b8c8-277dc7a0c203 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:18.638160 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 185/363] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:18.640100 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc886f-90e8-4471-9fbb-c8d96d905ae7 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:18.713891 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e5a50-5bc0-4d82-bf73-afad538a3983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:18.715433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e5a50-5bc0-4d82-bf73-afad538a3983 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:18.715900 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e5a50-5bc0-4d82-bf73-afad538a3983 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:18.716356 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e5a50-5bc0-4d82-bf73-afad538a3983 None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:18.720014 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e5a50-5bc0-4d82-bf73-afad538a3983 None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:18.721687 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e5a50-5bc0-4d82-bf73-afad538a3983 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:18.723986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 179/364] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:18 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:18.732713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e5a50-5bc0-4d82-bf73-afad538a3983 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:18.883474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adc886f-90e8-4471-9fbb-c8d96d905ae7 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:18.921180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e5a50-5bc0-4d82-bf73-afad538a3983 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:18.939894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67114606-5c79-4e63-b47b-6c8cc9cae9dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:18.940963 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67114606-5c79-4e63-b47b-6c8cc9cae9dd None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:18.941233 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67114606-5c79-4e63-b47b-6c8cc9cae9dd None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:18.941542 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67114606-5c79-4e63-b47b-6c8cc9cae9dd None admin] PATH_INFO: `/v3/users/f2d17980e771476a862bac8ea67be92a` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:18.942732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67114606-5c79-4e63-b47b-6c8cc9cae9dd None admin] RBAC: Authorizing `identity:get_user(user_id=f2d17980e771476a862bac8ea67be92a)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:18.944172 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67114606-5c79-4e63-b47b-6c8cc9cae9dd None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:18.946560 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 186/365] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 18:59:18 2022] GET /identity/v3/users/f2d17980e771476a862bac8ea67be92a => generated 293 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:18.947405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67114606-5c79-4e63-b47b-6c8cc9cae9dd None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:18.971689 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac493b-73b2-411c-90b5-868fc82d4800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:18.972329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac493b-73b2-411c-90b5-868fc82d4800 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:18.972610 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac493b-73b2-411c-90b5-868fc82d4800 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:18.972882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac493b-73b2-411c-90b5-868fc82d4800 None admin] PATH_INFO: `/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:18.974064 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac493b-73b2-411c-90b5-868fc82d4800 None admin] RBAC: Authorizing `identity:get_group(group_id=9aa3e6fe034d493bb846ba2e61112d0b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:18.975506 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac493b-73b2-411c-90b5-868fc82d4800 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:18.977293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 180/366] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 18:59:18 2022] GET /identity/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b => generated 226 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:18.982055 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac493b-73b2-411c-90b5-868fc82d4800 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:18.989666 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:19.010500 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:19.075705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67114606-5c79-4e63-b47b-6c8cc9cae9dd None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:19.092531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac493b-73b2-411c-90b5-868fc82d4800 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:19.115572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cccb39-ac0a-4f5c-ad8d-b56d2f741e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:19.116194 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cccb39-ac0a-4f5c-ad8d-b56d2f741e15 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:19.116464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cccb39-ac0a-4f5c-ad8d-b56d2f741e15 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:19.116726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cccb39-ac0a-4f5c-ad8d-b56d2f741e15 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:19.130118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fe41b-229f-4021-a951-89d0381a9e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:19.130724 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fe41b-229f-4021-a951-89d0381a9e0a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:19.131002 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fe41b-229f-4021-a951-89d0381a9e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:19.131269 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fe41b-229f-4021-a951-89d0381a9e0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:19.131369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cccb39-ac0a-4f5c-ad8d-b56d2f741e15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:19.131998 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fe41b-229f-4021-a951-89d0381a9e0a None admin] RBAC: Adding query filter params (role.id=2d35524d275d4bc4b09ce775ddec4963, user.id=f2d17980e771476a862bac8ea67be92a, scope.system=all) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:19.132304 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fe41b-229f-4021-a951-89d0381a9e0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:19.134834 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fe41b-229f-4021-a951-89d0381a9e0a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:19.136404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-07cccb39-ac0a-4f5c-ad8d-b56d2f741e15 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:59:19.136404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:19.136404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:19.136404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:19.136404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:19.136404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:19.136404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:19.136404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:19.136404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:19.136404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:19.136404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:19.136404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:19.136404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:19.136404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:19.136404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:19.136404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:19.137286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:19.137286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:19.137286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:19.137286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:19.137286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:19.137286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:19.137286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:19.137286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:19.137286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:19.137286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:19.137286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:19.137286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:19.137286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:19.137286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:19.137286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:19.137286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:19.138094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:19.138094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:19.138094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:19.138094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:19.138094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:19.138094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:19.138094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:19.138094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:19.138094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:19.138094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:19.138094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:59:19.138094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:19.138094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 187/367] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 18:59:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:19.139332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cccb39-ac0a-4f5c-ad8d-b56d2f741e15 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:19.151970 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 181/368] 10.222.0.65 () {56 vars in 1308 bytes} [Wed Aug 31 18:59:18 2022] GET /identity/v3/role_assignments?user.id=f2d17980e771476a862bac8ea67be92a&scope.system=all&role.id=2d35524d275d4bc4b09ce775ddec4963 => generated 510 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:19.153815 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fe41b-229f-4021-a951-89d0381a9e0a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:19.227654 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:19.272831 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fe41b-229f-4021-a951-89d0381a9e0a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:19.309975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702047ca-68c7-47c7-acec-5f34dd0841ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:19.310588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702047ca-68c7-47c7-acec-5f34dd0841ca None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:19.310987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702047ca-68c7-47c7-acec-5f34dd0841ca None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:19.311262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702047ca-68c7-47c7-acec-5f34dd0841ca None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:19.312052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702047ca-68c7-47c7-acec-5f34dd0841ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:19.312356 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702047ca-68c7-47c7-acec-5f34dd0841ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:19.313777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702047ca-68c7-47c7-acec-5f34dd0841ca None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:19.323402 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 182/369] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:19.324217 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702047ca-68c7-47c7-acec-5f34dd0841ca None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:19.326905 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cccb39-ac0a-4f5c-ad8d-b56d2f741e15 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:19.336252 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:19.371219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58250250-9109-4dc4-8372-3339e8292703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:19.371911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58250250-9109-4dc4-8372-3339e8292703 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:19.372193 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58250250-9109-4dc4-8372-3339e8292703 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:19.372453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58250250-9109-4dc4-8372-3339e8292703 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:19.373561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58250250-9109-4dc4-8372-3339e8292703 None admin] RBAC: Adding query filter params (group.id=9aa3e6fe034d493bb846ba2e61112d0b, role.id=b80a0d1a8d164b53bef5726b92e68676, scope.project.id=8ab3b89f1290419e93f22ea7eee77da8) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:19.373861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58250250-9109-4dc4-8372-3339e8292703 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:19.375353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58250250-9109-4dc4-8372-3339e8292703 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:19.382010 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 188/370] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 18:59:19 2022] GET /identity/v3/role_assignments?group.id=9aa3e6fe034d493bb846ba2e61112d0b&scope.project.id=8ab3b89f1290419e93f22ea7eee77da8&role.id=b80a0d1a8d164b53bef5726b92e68676 => generated 260 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:19.383020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58250250-9109-4dc4-8372-3339e8292703 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:19.506547 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702047ca-68c7-47c7-acec-5f34dd0841ca None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:19.521711 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58250250-9109-4dc4-8372-3339e8292703 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:19.551025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:19.571324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358b48b-f713-4ece-a529-08f1f834f9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:19.571951 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358b48b-f713-4ece-a529-08f1f834f9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:19.572226 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358b48b-f713-4ece-a529-08f1f834f9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:19.572489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358b48b-f713-4ece-a529-08f1f834f9c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:19.575452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ae918-4196-44f5-b369-8df39013b370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:19.576210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ae918-4196-44f5-b369-8df39013b370 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:19.576495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ae918-4196-44f5-b369-8df39013b370 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:19.576769 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ae918-4196-44f5-b369-8df39013b370 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:19.606884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449ae918-4196-44f5-b369-8df39013b370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:19.612386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-449ae918-4196-44f5-b369-8df39013b370 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:59:19.612386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:19.612386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:19.612386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:19.612386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:19.612386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:19.612386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:19.612386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:19.612386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:19.612386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:19.612386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:19.612386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:19.612386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:19.612386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:19.612386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:19.612386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:19.613369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:19.613369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:19.613369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:19.613369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:19.613369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:19.613369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:19.613369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:19.613369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:19.613369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:19.613369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:19.613369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:19.613369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:19.613369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:19.613369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:19.613369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:19.613369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:19.614152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:19.614152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:19.614152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:19.614152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:19.614152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:19.614152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:19.614152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:19.614152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:19.614152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:19.614152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:19.614152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:59:19.614152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:19.614152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 183/371] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:59:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:19.634927 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ae918-4196-44f5-b369-8df39013b370 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:19.657497 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:19.711392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358b48b-f713-4ece-a529-08f1f834f9c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:19.712868 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358b48b-f713-4ece-a529-08f1f834f9c2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:19.780857 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ae918-4196-44f5-b369-8df39013b370 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:19.830315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307a88c-8a1c-43f5-bbda-ffa04590a814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:19.831301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307a88c-8a1c-43f5-bbda-ffa04590a814 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:19.831778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307a88c-8a1c-43f5-bbda-ffa04590a814 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:19.832246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b307a88c-8a1c-43f5-bbda-ffa04590a814 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:19.833674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307a88c-8a1c-43f5-bbda-ffa04590a814 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:19.834450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307a88c-8a1c-43f5-bbda-ffa04590a814 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:19.836729 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b307a88c-8a1c-43f5-bbda-ffa04590a814 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:19.848709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 184/373] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:19.848888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 189/373] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:19.854297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358b48b-f713-4ece-a529-08f1f834f9c2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:19.855056 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307a88c-8a1c-43f5-bbda-ffa04590a814 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:19.916167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:19.950600 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:19.987505 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358b48b-f713-4ece-a529-08f1f834f9c2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:19.992445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b307a88c-8a1c-43f5-bbda-ffa04590a814 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:20.032883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190f828-46fb-4975-8f7b-d6b63388282a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:20.033505 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190f828-46fb-4975-8f7b-d6b63388282a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:20.033895 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190f828-46fb-4975-8f7b-d6b63388282a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:20.034263 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190f828-46fb-4975-8f7b-d6b63388282a None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:20.035669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190f828-46fb-4975-8f7b-d6b63388282a None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:20.037165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190f828-46fb-4975-8f7b-d6b63388282a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:20.039803 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 190/374] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:19 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:20.044037 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4b776-c1a6-4e86-9f32-0bb601daa76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:20.044823 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4b776-c1a6-4e86-9f32-0bb601daa76d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:20.045279 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4b776-c1a6-4e86-9f32-0bb601daa76d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:20.045686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca4b776-c1a6-4e86-9f32-0bb601daa76d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:20.046030 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190f828-46fb-4975-8f7b-d6b63388282a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:20.137552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4b776-c1a6-4e86-9f32-0bb601daa76d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:20.139497 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca4b776-c1a6-4e86-9f32-0bb601daa76d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:20.163266 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190f828-46fb-4975-8f7b-d6b63388282a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:20.205221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:20.211903 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7add72-73d0-41a5-991a-9f1da169ac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:20.212546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7add72-73d0-41a5-991a-9f1da169ac72 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:20.212821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7add72-73d0-41a5-991a-9f1da169ac72 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:20.214080 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7add72-73d0-41a5-991a-9f1da169ac72 None admin] PATH_INFO: `/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:20.219020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7add72-73d0-41a5-991a-9f1da169ac72 None admin] RBAC: Authorizing `identity:get_group(group_id=9aa3e6fe034d493bb846ba2e61112d0b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:20.220026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7add72-73d0-41a5-991a-9f1da169ac72 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:20.222923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 191/375] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 18:59:20 2022] GET /identity/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:20.230501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7add72-73d0-41a5-991a-9f1da169ac72 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:20.242788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:20.283264 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 185/376] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 18:59:20.293257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4b776-c1a6-4e86-9f32-0bb601daa76d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:20.373008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7add72-73d0-41a5-991a-9f1da169ac72 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:20.409454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca4b776-c1a6-4e86-9f32-0bb601daa76d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:20.419197 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee91e4b-a3b7-4137-af39-c167690dabca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:20.420046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee91e4b-a3b7-4137-af39-c167690dabca None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:20.420489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee91e4b-a3b7-4137-af39-c167690dabca None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:20.421494 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee91e4b-a3b7-4137-af39-c167690dabca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:20.422367 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee91e4b-a3b7-4137-af39-c167690dabca None admin] RBAC: Adding query filter params (group.id=9aa3e6fe034d493bb846ba2e61112d0b, role.id=470b660efbe04083a803e79d8b647160, scope.project.id=8ab3b89f1290419e93f22ea7eee77da8) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:20.422814 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee91e4b-a3b7-4137-af39-c167690dabca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:20.425334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee91e4b-a3b7-4137-af39-c167690dabca None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:20.433263 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 192/377] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 18:59:20 2022] GET /identity/v3/role_assignments?group.id=9aa3e6fe034d493bb846ba2e61112d0b&scope.project.id=8ab3b89f1290419e93f22ea7eee77da8&role.id=470b660efbe04083a803e79d8b647160 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:20.456883 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd0f34-0811-40fa-93e6-4919c57e0209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:20.457868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd0f34-0811-40fa-93e6-4919c57e0209 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:20.458283 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd0f34-0811-40fa-93e6-4919c57e0209 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:20.458712 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd0f34-0811-40fa-93e6-4919c57e0209 None admin] PATH_INFO: `/v3/projects/8d8519390e3f4536964a76bc41b6e43c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:20.460024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd0f34-0811-40fa-93e6-4919c57e0209 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8519390e3f4536964a76bc41b6e43c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:20.461656 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd0f34-0811-40fa-93e6-4919c57e0209 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:20.463787 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 186/378] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:20 2022] GET /identity/v3/projects/8d8519390e3f4536964a76bc41b6e43c => generated 344 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:20.470357 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee91e4b-a3b7-4137-af39-c167690dabca None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:20.551178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:20.586159 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee91e4b-a3b7-4137-af39-c167690dabca None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:20.624165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10108cd4-5432-4a5e-99f1-3a97fca87519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:20.625050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10108cd4-5432-4a5e-99f1-3a97fca87519 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:20.625509 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10108cd4-5432-4a5e-99f1-3a97fca87519 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:20.625921 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10108cd4-5432-4a5e-99f1-3a97fca87519 None admin] PATH_INFO: `/v3/groups/41f40db10ab7437abd9c4505024abf7b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:20.631513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10108cd4-5432-4a5e-99f1-3a97fca87519 None admin] RBAC: Authorizing `identity:get_group(group_id=41f40db10ab7437abd9c4505024abf7b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:20.633143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10108cd4-5432-4a5e-99f1-3a97fca87519 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:20.634952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 193/379] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 18:59:20 2022] GET /identity/v3/groups/41f40db10ab7437abd9c4505024abf7b => generated 229 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:20.641569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd0f34-0811-40fa-93e6-4919c57e0209 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:20.658325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:20.751627 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd0f34-0811-40fa-93e6-4919c57e0209 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:20.788483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d242ec-1268-4a12-a39a-bdf9c434c981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:20.789401 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d242ec-1268-4a12-a39a-bdf9c434c981 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:20.789857 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d242ec-1268-4a12-a39a-bdf9c434c981 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:20.790300 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d242ec-1268-4a12-a39a-bdf9c434c981 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:20.791274 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d242ec-1268-4a12-a39a-bdf9c434c981 None admin] RBAC: Adding query filter params (group.id=41f40db10ab7437abd9c4505024abf7b, role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=8d8519390e3f4536964a76bc41b6e43c) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:20.791754 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d242ec-1268-4a12-a39a-bdf9c434c981 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:20.793411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d242ec-1268-4a12-a39a-bdf9c434c981 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:20.799415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 187/380] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 18:59:20 2022] GET /identity/v3/role_assignments?group.id=41f40db10ab7437abd9c4505024abf7b&scope.project.id=8d8519390e3f4536964a76bc41b6e43c&role.id=3580c894dd974d35bc95848b8ba22598 => generated 260 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:20.866150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d704aa-db94-4f73-9c7d-295c3b80fc2f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:20.866664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d704aa-db94-4f73-9c7d-295c3b80fc2f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:20.866664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d704aa-db94-4f73-9c7d-295c3b80fc2f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:20.868456 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 194/381] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:20.874725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33421f-55e0-4566-b9ba-49dd66fe8069 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:20.875263 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33421f-55e0-4566-b9ba-49dd66fe8069 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:20.875700 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff33421f-55e0-4566-b9ba-49dd66fe8069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:20.895936 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ff33421f-55e0-4566-b9ba-49dd66fe8069 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:20.897464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ff33421f-55e0-4566-b9ba-49dd66fe8069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:20.978700 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 188/382] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:21.148429 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84e584-9684-4a69-83d2-62900e12d84d None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:21.148771 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84e584-9684-4a69-83d2-62900e12d84d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:21.148890 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84e584-9684-4a69-83d2-62900e12d84d None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:21.150527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 195/383] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:21.157425 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c296f-6911-4dc0-a951-1f52951b7e18 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:21.157957 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c296f-6911-4dc0-a951-1f52951b7e18 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:21.158421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c296f-6911-4dc0-a951-1f52951b7e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:21.172656 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:21.179975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-106c296f-6911-4dc0-a951-1f52951b7e18 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:21.181425 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-106c296f-6911-4dc0-a951-1f52951b7e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:21.271655 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 189/384] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:21.328042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84e584-9684-4a69-83d2-62900e12d84d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:21.451459 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84e584-9684-4a69-83d2-62900e12d84d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:21.491616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c290f5b-269c-4f2a-854e-ddabb1692be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:21.492670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c290f5b-269c-4f2a-854e-ddabb1692be8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:21.493169 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c290f5b-269c-4f2a-854e-ddabb1692be8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:21.493691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c290f5b-269c-4f2a-854e-ddabb1692be8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:21.496304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:21.508059 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c290f5b-269c-4f2a-854e-ddabb1692be8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:21.513448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-6c290f5b-269c-4f2a-854e-ddabb1692be8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:21.513448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:21.513448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:21.513448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:21.513448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:21.513448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:21.513448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:21.513448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:21.513448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:21.513448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:21.513448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:21.513448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:21.513448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:21.513448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:21.513448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:21.513448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:21.514319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:21.514319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:21.514319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:21.514319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:21.514319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:21.514319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:21.514319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:21.514319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:21.514319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:21.514319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:21.514319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:21.514319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:21.514319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:21.514319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:21.514319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:21.514319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:21.515101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:21.515101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:21.515101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:21.515101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:21.515101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:21.515101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:21.515101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:21.515101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:21.515101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:21.515101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:21.515101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:21.515101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:21.517698 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 196/385] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:21.523863 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c296f-6911-4dc0-a951-1f52951b7e18 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:21.651177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c296f-6911-4dc0-a951-1f52951b7e18 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:21.686183 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c290f5b-269c-4f2a-854e-ddabb1692be8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:21.688798 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147feec9-6ed4-4165-b0a3-fa491348b864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:21.690606 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147feec9-6ed4-4165-b0a3-fa491348b864 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:21.690889 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147feec9-6ed4-4165-b0a3-fa491348b864 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:21.691155 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147feec9-6ed4-4165-b0a3-fa491348b864 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:21.691880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147feec9-6ed4-4165-b0a3-fa491348b864 None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:21.692435 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147feec9-6ed4-4165-b0a3-fa491348b864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:21.694795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147feec9-6ed4-4165-b0a3-fa491348b864 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:21.697224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:21.710422 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 190/386] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:21.718799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147feec9-6ed4-4165-b0a3-fa491348b864 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:21.872369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c290f5b-269c-4f2a-854e-ddabb1692be8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:21.891321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147feec9-6ed4-4165-b0a3-fa491348b864 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:21.946297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc30c64-ed33-4d14-bc7e-295dce638671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:21.946934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc30c64-ed33-4d14-bc7e-295dce638671 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:21.947209 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc30c64-ed33-4d14-bc7e-295dce638671 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:21.947478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc30c64-ed33-4d14-bc7e-295dce638671 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:21.961499 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af6804-4f33-4f18-a9ad-fb2da2ea2f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:21.962111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af6804-4f33-4f18-a9ad-fb2da2ea2f95 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:21.962352 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af6804-4f33-4f18-a9ad-fb2da2ea2f95 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:21.962633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79af6804-4f33-4f18-a9ad-fb2da2ea2f95 None admin] PATH_INFO: `/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:21.965469 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af6804-4f33-4f18-a9ad-fb2da2ea2f95 None admin] RBAC: Authorizing `identity:get_group(group_id=9aa3e6fe034d493bb846ba2e61112d0b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:21.966991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79af6804-4f33-4f18-a9ad-fb2da2ea2f95 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:21.968755 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 191/387] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 18:59:21 2022] GET /identity/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b => generated 226 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:21.978119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc30c64-ed33-4d14-bc7e-295dce638671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:21.978511 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af6804-4f33-4f18-a9ad-fb2da2ea2f95 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:21.985478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-2bc30c64-ed33-4d14-bc7e-295dce638671 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:59:21.985478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:21.985478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:21.985478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:21.985478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:21.985478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:21.985478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:21.985478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:21.985478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:21.985478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:21.985478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:21.985478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:21.985478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:21.985478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:21.985478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:21.985478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:21.986361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:21.986361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:21.986361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:21.986361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:21.986361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:21.986361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:21.986361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:21.986361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:21.986361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:21.986361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:21.986361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:21.986361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:21.986361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:21.986361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:21.986361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:21.986361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:21.987143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:21.987143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:21.987143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:21.987143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:21.987143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:21.987143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:21.987143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:21.987143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:21.987143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:21.987143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:21.987143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:59:21.987143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:21.987143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 197/388] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 18:59:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:21.997512 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc30c64-ed33-4d14-bc7e-295dce638671 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:22.078593 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:22.086176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:22.131709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79af6804-4f33-4f18-a9ad-fb2da2ea2f95 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:22.143685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc30c64-ed33-4d14-bc7e-295dce638671 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:22.185327 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200a123-e01e-4e3e-9111-37ddedd3c941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:22.186652 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200a123-e01e-4e3e-9111-37ddedd3c941 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:22.187249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200a123-e01e-4e3e-9111-37ddedd3c941 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:22.187757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f200a123-e01e-4e3e-9111-37ddedd3c941 None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:22.189153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200a123-e01e-4e3e-9111-37ddedd3c941 None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:22.197008 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f200a123-e01e-4e3e-9111-37ddedd3c941 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:22.199994 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 192/389] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:21 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:22.205927 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200a123-e01e-4e3e-9111-37ddedd3c941 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:22.216210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f600ef-a7f0-4c45-bede-9df47d713dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:22.216761 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f600ef-a7f0-4c45-bede-9df47d713dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:22.217037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f600ef-a7f0-4c45-bede-9df47d713dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:22.217340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f600ef-a7f0-4c45-bede-9df47d713dd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:22.218008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f600ef-a7f0-4c45-bede-9df47d713dd9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:22.218308 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f600ef-a7f0-4c45-bede-9df47d713dd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:22.219690 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f600ef-a7f0-4c45-bede-9df47d713dd9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:22.232576 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 198/390] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:22.247965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f600ef-a7f0-4c45-bede-9df47d713dd9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:22.375196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f600ef-a7f0-4c45-bede-9df47d713dd9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:22.379130 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f200a123-e01e-4e3e-9111-37ddedd3c941 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:22.405628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:22.415836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eb70f4-79a8-4779-a112-8f7d0e916e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:22.416491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb70f4-79a8-4779-a112-8f7d0e916e8e None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:22.416782 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb70f4-79a8-4779-a112-8f7d0e916e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:22.417059 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eb70f4-79a8-4779-a112-8f7d0e916e8e None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4/groups/9aa3e6fe034d493bb846ba2e61112d0b/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:22.418799 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adad3d1-5d9c-42aa-a26c-76046808a611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:22.419429 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb70f4-79a8-4779-a112-8f7d0e916e8e None admin] RBAC: Authorizing `identity:create_grant(project_id=7d21b4b8971a47ff8985309015eb53d4, group_id=9aa3e6fe034d493bb846ba2e61112d0b, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:22.419635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adad3d1-5d9c-42aa-a26c-76046808a611 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:22.420011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adad3d1-5d9c-42aa-a26c-76046808a611 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:22.420448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adad3d1-5d9c-42aa-a26c-76046808a611 None admin] PATH_INFO: `/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:22.420851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eb70f4-79a8-4779-a112-8f7d0e916e8e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:22.421813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adad3d1-5d9c-42aa-a26c-76046808a611 None admin] RBAC: Authorizing `identity:get_group(group_id=9aa3e6fe034d493bb846ba2e61112d0b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:22.423308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:22.425573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adad3d1-5d9c-42aa-a26c-76046808a611 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:22.432414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 199/391] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 18:59:22 2022] GET /identity/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:22.432414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035076b-9f0b-423e-9b0b-5f33a55cb92c None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:22.432414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035076b-9f0b-423e-9b0b-5f33a55cb92c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:22.432414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a035076b-9f0b-423e-9b0b-5f33a55cb92c None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:22.432747 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 200/392] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:22.434932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035076b-9f0b-423e-9b0b-5f33a55cb92c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:22.448237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 193/393] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 31 18:59:22 2022] PUT /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4/groups/9aa3e6fe034d493bb846ba2e61112d0b/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 18:59:22.453843 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0763fa2-7ce9-44da-924b-0be979940385 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:22.455171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0763fa2-7ce9-44da-924b-0be979940385 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:22.455439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0763fa2-7ce9-44da-924b-0be979940385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:22.484680 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e0763fa2-7ce9-44da-924b-0be979940385 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:22.487098 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e0763fa2-7ce9-44da-924b-0be979940385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:22.595094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a035076b-9f0b-423e-9b0b-5f33a55cb92c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:22.596662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 194/394] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:22.640994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284e0f4-fab2-451e-9ba9-1d9d04ddcda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:22.642031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284e0f4-fab2-451e-9ba9-1d9d04ddcda3 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:22.642531 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284e0f4-fab2-451e-9ba9-1d9d04ddcda3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:22.643025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f284e0f4-fab2-451e-9ba9-1d9d04ddcda3 None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:22.644416 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284e0f4-fab2-451e-9ba9-1d9d04ddcda3 None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:22.646111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f284e0f4-fab2-451e-9ba9-1d9d04ddcda3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:22.648362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 201/395] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:22 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:22.654465 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0763fa2-7ce9-44da-924b-0be979940385 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:22.774760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0763fa2-7ce9-44da-924b-0be979940385 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:22.805392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:22.822957 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3208133c-f180-4ad8-b608-879c7e3cfaae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:22.823611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208133c-f180-4ad8-b608-879c7e3cfaae None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:22.824432 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208133c-f180-4ad8-b608-879c7e3cfaae None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:22.824706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3208133c-f180-4ad8-b608-879c7e3cfaae None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4/groups/9aa3e6fe034d493bb846ba2e61112d0b/roles/470b660efbe04083a803e79d8b647160` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:22.828606 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208133c-f180-4ad8-b608-879c7e3cfaae None admin] RBAC: Authorizing `identity:create_grant(project_id=7d21b4b8971a47ff8985309015eb53d4, group_id=9aa3e6fe034d493bb846ba2e61112d0b, role_id=470b660efbe04083a803e79d8b647160)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:22.830095 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3208133c-f180-4ad8-b608-879c7e3cfaae None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:22.854907 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 195/396] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 31 18:59:22 2022] PUT /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4/groups/9aa3e6fe034d493bb846ba2e61112d0b/roles/470b660efbe04083a803e79d8b647160 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 18:59:22.951689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284e0f4-fab2-451e-9ba9-1d9d04ddcda3 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:23.023628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:23.059028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9d8b6-e1b6-437d-8b02-0997256e8b86 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:23.059208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9d8b6-e1b6-437d-8b02-0997256e8b86 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:23.059480 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9d8b6-e1b6-437d-8b02-0997256e8b86 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:23.060949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 196/397] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:23.070597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb9d4a-0596-41df-9542-fc63a9986f61 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:23.070890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb9d4a-0596-41df-9542-fc63a9986f61 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:23.071146 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cb9d4a-0596-41df-9542-fc63a9986f61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:23.088685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f284e0f4-fab2-451e-9ba9-1d9d04ddcda3 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:23.114301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-63cb9d4a-0596-41df-9542-fc63a9986f61 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:23.115525 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-63cb9d4a-0596-41df-9542-fc63a9986f61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:23.130142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91ba85-b145-4064-a6c6-2b535a56c164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:23.130782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91ba85-b145-4064-a6c6-2b535a56c164 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:23.131061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91ba85-b145-4064-a6c6-2b535a56c164 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:23.131325 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91ba85-b145-4064-a6c6-2b535a56c164 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:23.143128 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91ba85-b145-4064-a6c6-2b535a56c164 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:23.148628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-4e91ba85-b145-4064-a6c6-2b535a56c164 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:23.148628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:23.148628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:23.148628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:23.148628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:23.148628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:23.148628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:23.148628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:23.148628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:23.148628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:23.148628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:23.148628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:23.148628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:23.148628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:23.148628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:23.148628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:23.149539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:23.149539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:23.149539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:23.149539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:23.149539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:23.149539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:23.149539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:23.149539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:23.149539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:23.149539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:23.149539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:23.149539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:23.149539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:23.149539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:23.149539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:23.149539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:23.150324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:23.150324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:23.150324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:23.150324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:23.150324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:23.150324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:23.150324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:23.150324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:23.150324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:23.150324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:23.150324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:23.150324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:23.150324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 202/398] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:23.155999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91ba85-b145-4064-a6c6-2b535a56c164 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:23.223297 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 197/399] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:59:23.281166 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91ba85-b145-4064-a6c6-2b535a56c164 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:23.295616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:23.318309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19395d-abdd-4db9-85d1-eb26e7b9bc47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:23.319185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19395d-abdd-4db9-85d1-eb26e7b9bc47 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:23.319714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19395d-abdd-4db9-85d1-eb26e7b9bc47 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:23.320163 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19395d-abdd-4db9-85d1-eb26e7b9bc47 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:23.320996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19395d-abdd-4db9-85d1-eb26e7b9bc47 None admin] RBAC: Adding query filter params (name=member) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:23.321491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19395d-abdd-4db9-85d1-eb26e7b9bc47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:23.323151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af19395d-abdd-4db9-85d1-eb26e7b9bc47 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:23.332405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 203/400] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:23.338238 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb9d4a-0596-41df-9542-fc63a9986f61 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:23.444454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cb9d4a-0596-41df-9542-fc63a9986f61 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:23.459019 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:23.490329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a7044-adaa-49c3-bd71-f339d934a2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:23.491370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a7044-adaa-49c3-bd71-f339d934a2ec None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:23.491828 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a7044-adaa-49c3-bd71-f339d934a2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:23.492322 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a7044-adaa-49c3-bd71-f339d934a2ec None admin] PATH_INFO: `/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:23.493635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a7044-adaa-49c3-bd71-f339d934a2ec None admin] RBAC: Authorizing `identity:get_group(group_id=9aa3e6fe034d493bb846ba2e61112d0b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:23.495286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a7044-adaa-49c3-bd71-f339d934a2ec None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:23.497089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 198/401] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 18:59:23 2022] GET /identity/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:23.503072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19395d-abdd-4db9-85d1-eb26e7b9bc47 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:23.532339 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd24607-1e25-48e7-b754-8476789d3082 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:23.533159 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd24607-1e25-48e7-b754-8476789d3082 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:23.533752 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd24607-1e25-48e7-b754-8476789d3082 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:23.535541 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 199/402] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:23.540659 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062001b-4f38-4cc4-813e-f296fe531d4d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:23.541141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062001b-4f38-4cc4-813e-f296fe531d4d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:23.542107 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2062001b-4f38-4cc4-813e-f296fe531d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:23.566242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2062001b-4f38-4cc4-813e-f296fe531d4d None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:23.568285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2062001b-4f38-4cc4-813e-f296fe531d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:23.628111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19395d-abdd-4db9-85d1-eb26e7b9bc47 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:23.660355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 200/403] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:23.661862 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062001b-4f38-4cc4-813e-f296fe531d4d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:23.672419 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2f985-3814-46d5-8f3a-5c3c82ad83b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:23.673109 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2f985-3814-46d5-8f3a-5c3c82ad83b7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:23.673478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2f985-3814-46d5-8f3a-5c3c82ad83b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:23.673751 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2f985-3814-46d5-8f3a-5c3c82ad83b7 None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:23.676029 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a2f985-3814-46d5-8f3a-5c3c82ad83b7 None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:23.677546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a2f985-3814-46d5-8f3a-5c3c82ad83b7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:23.679539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 204/404] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:23 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:23.685644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2f985-3814-46d5-8f3a-5c3c82ad83b7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:23.769361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:23.792950 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2062001b-4f38-4cc4-813e-f296fe531d4d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:23.808767 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:23.859386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c78dbc-2797-41aa-aa77-82969b011efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:23.860395 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c78dbc-2797-41aa-aa77-82969b011efb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:23.860870 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c78dbc-2797-41aa-aa77-82969b011efb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:23.862508 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c78dbc-2797-41aa-aa77-82969b011efb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:23.883485 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2f985-3814-46d5-8f3a-5c3c82ad83b7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:23.887034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c78dbc-2797-41aa-aa77-82969b011efb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:23.892172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-63c78dbc-2797-41aa-aa77-82969b011efb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:59:23.892172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:23.892172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:23.892172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:23.892172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:23.892172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:23.892172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:23.892172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:23.892172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:23.892172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:23.892172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:23.892172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:23.892172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:23.892172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:23.892172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:23.892172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:23.893020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:23.893020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:23.893020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:23.893020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:23.893020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:23.893020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:23.893020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:23.893020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:23.893020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:23.893020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:23.893020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:23.893020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:23.893020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:23.893020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:23.893020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:23.893020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:23.893877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:23.893877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:23.893877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:23.893877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:23.893877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:23.893877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:23.893877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:23.893877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:23.893877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:23.893877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:23.893877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:59:23.893877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:23.897579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 201/405] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 18:59:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:23.904476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c78dbc-2797-41aa-aa77-82969b011efb None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:23.939129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327a6b6-ced8-47ea-b75b-6a3dbd0ef4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:23.940206 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327a6b6-ced8-47ea-b75b-6a3dbd0ef4cc None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:23.940689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327a6b6-ced8-47ea-b75b-6a3dbd0ef4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:23.941243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327a6b6-ced8-47ea-b75b-6a3dbd0ef4cc None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8/groups/9aa3e6fe034d493bb846ba2e61112d0b/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:23.943472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327a6b6-ced8-47ea-b75b-6a3dbd0ef4cc None admin] RBAC: Authorizing `identity:create_grant(project_id=8ab3b89f1290419e93f22ea7eee77da8, group_id=9aa3e6fe034d493bb846ba2e61112d0b, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:23.945228 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327a6b6-ced8-47ea-b75b-6a3dbd0ef4cc None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:23.963573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 205/406] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 31 18:59:23 2022] PUT /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8/groups/9aa3e6fe034d493bb846ba2e61112d0b/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 18:59:24.051379 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327a6b6-ced8-47ea-b75b-6a3dbd0ef4cc None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:24.054249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c78dbc-2797-41aa-aa77-82969b011efb None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:24.097817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57481f96-09ae-415c-8652-b72b8032a9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:24.098798 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57481f96-09ae-415c-8652-b72b8032a9de None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:24.099289 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57481f96-09ae-415c-8652-b72b8032a9de None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:24.099730 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57481f96-09ae-415c-8652-b72b8032a9de None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:24.100578 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57481f96-09ae-415c-8652-b72b8032a9de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:24.101052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57481f96-09ae-415c-8652-b72b8032a9de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:24.103366 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57481f96-09ae-415c-8652-b72b8032a9de None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:24.114043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 202/407] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:24.121262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57481f96-09ae-415c-8652-b72b8032a9de None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:24.169665 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:24.216896 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327a6b6-ced8-47ea-b75b-6a3dbd0ef4cc None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:24.218966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:24.262664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699572d-82c2-489f-8a85-af41a0ff2404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:24.263891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699572d-82c2-489f-8a85-af41a0ff2404 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:24.264782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699572d-82c2-489f-8a85-af41a0ff2404 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:24.265291 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8699572d-82c2-489f-8a85-af41a0ff2404 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:24.280837 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57481f96-09ae-415c-8652-b72b8032a9de None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:24.282007 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8699572d-82c2-489f-8a85-af41a0ff2404 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:24.288873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-8699572d-82c2-489f-8a85-af41a0ff2404 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:59:24.288873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:24.288873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:24.288873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:24.288873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:24.288873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:24.288873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:24.288873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:24.288873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:24.288873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:24.288873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:24.288873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:24.288873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:24.288873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:24.288873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:24.288873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:24.289873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:24.289873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:24.289873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:24.289873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:24.289873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:24.289873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:24.289873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:24.289873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:24.289873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:24.289873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:24.289873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:24.289873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:24.289873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:24.289873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:24.289873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:24.289873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:24.290727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:24.290727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:24.290727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:24.290727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:24.290727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:24.290727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:24.290727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:24.290727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:24.290727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:24.290727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:24.290727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:59:24.290727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:24.293959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 206/408] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:59:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 18:59:24.299199 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699572d-82c2-489f-8a85-af41a0ff2404 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:24.342680 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a857581-bf79-4bdf-a23c-8c14d136d962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:24.343403 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a857581-bf79-4bdf-a23c-8c14d136d962 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:24.343745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a857581-bf79-4bdf-a23c-8c14d136d962 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:24.344079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a857581-bf79-4bdf-a23c-8c14d136d962 None admin] PATH_INFO: `/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:24.346244 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a857581-bf79-4bdf-a23c-8c14d136d962 None admin] RBAC: Authorizing `identity:get_group(group_id=9aa3e6fe034d493bb846ba2e61112d0b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:24.349591 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a857581-bf79-4bdf-a23c-8c14d136d962 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:24.351977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 203/409] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 18:59:24 2022] GET /identity/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b => generated 226 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:24.357442 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a857581-bf79-4bdf-a23c-8c14d136d962 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:24.454575 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8699572d-82c2-489f-8a85-af41a0ff2404 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:24.505429 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73bc31-a9b2-4e90-8faa-ed87cb0aec21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:24.506424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73bc31-a9b2-4e90-8faa-ed87cb0aec21 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:24.506910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73bc31-a9b2-4e90-8faa-ed87cb0aec21 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:24.507386 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73bc31-a9b2-4e90-8faa-ed87cb0aec21 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:24.508241 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73bc31-a9b2-4e90-8faa-ed87cb0aec21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:24.508757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73bc31-a9b2-4e90-8faa-ed87cb0aec21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:24.510349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73bc31-a9b2-4e90-8faa-ed87cb0aec21 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:24.519365 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 207/410] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:24.525065 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73bc31-a9b2-4e90-8faa-ed87cb0aec21 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:24.530941 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a857581-bf79-4bdf-a23c-8c14d136d962 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:24.532392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:24.557886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:24.576175 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771e627-8031-4ea6-88ee-63f93877dbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:24.577042 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771e627-8031-4ea6-88ee-63f93877dbef None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:24.577588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771e627-8031-4ea6-88ee-63f93877dbef None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:24.578001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771e627-8031-4ea6-88ee-63f93877dbef None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:24.579370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771e627-8031-4ea6-88ee-63f93877dbef None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:24.581049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771e627-8031-4ea6-88ee-63f93877dbef None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:24.583074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 204/411] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:24 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:24.588809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771e627-8031-4ea6-88ee-63f93877dbef None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:24.696013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73bc31-a9b2-4e90-8faa-ed87cb0aec21 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:24.735042 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771e627-8031-4ea6-88ee-63f93877dbef None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:24.773677 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb38c26-3d6e-4322-8627-454271d40008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:24.774307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb38c26-3d6e-4322-8627-454271d40008 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:24.774584 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb38c26-3d6e-4322-8627-454271d40008 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:24.774854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb38c26-3d6e-4322-8627-454271d40008 None admin] PATH_INFO: `/v3/groups/41f40db10ab7437abd9c4505024abf7b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:24.776296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb38c26-3d6e-4322-8627-454271d40008 None admin] RBAC: Authorizing `identity:get_group(group_id=41f40db10ab7437abd9c4505024abf7b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:24.778783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb38c26-3d6e-4322-8627-454271d40008 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:24.780383 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 208/412] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 18:59:24 2022] GET /identity/v3/groups/41f40db10ab7437abd9c4505024abf7b => generated 229 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:24.788399 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb38c26-3d6e-4322-8627-454271d40008 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:24.829102 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a513d-4d19-485d-9a38-df56611ac47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:24.829825 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a513d-4d19-485d-9a38-df56611ac47c None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:24.830106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a513d-4d19-485d-9a38-df56611ac47c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:24.830382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a513d-4d19-485d-9a38-df56611ac47c None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8/groups/9aa3e6fe034d493bb846ba2e61112d0b/roles/470b660efbe04083a803e79d8b647160` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:24.832540 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a513d-4d19-485d-9a38-df56611ac47c None admin] RBAC: Authorizing `identity:create_grant(project_id=8ab3b89f1290419e93f22ea7eee77da8, group_id=9aa3e6fe034d493bb846ba2e61112d0b, role_id=470b660efbe04083a803e79d8b647160)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:24.833992 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a513d-4d19-485d-9a38-df56611ac47c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:24.876594 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 205/413] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 31 18:59:24 2022] PUT /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8/groups/9aa3e6fe034d493bb846ba2e61112d0b/roles/470b660efbe04083a803e79d8b647160 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 18:59:24.881693 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f66ea-2575-4c20-9558-a83d30e05681 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:24.882032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f66ea-2575-4c20-9558-a83d30e05681 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:24.882312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66f66ea-2575-4c20-9558-a83d30e05681 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:24.883725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 206/414] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:24.892468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:24.899946 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1fcfb-0c53-44b1-bd03-1648dc46efce None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:24.900277 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1fcfb-0c53-44b1-bd03-1648dc46efce None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:24.900493 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1fcfb-0c53-44b1-bd03-1648dc46efce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:24.902579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb38c26-3d6e-4322-8627-454271d40008 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:24.941368 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340ab66-8f74-40be-9baf-75c3c1701d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:24.942013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340ab66-8f74-40be-9baf-75c3c1701d21 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:24.942294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340ab66-8f74-40be-9baf-75c3c1701d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:24.942732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7340ab66-8f74-40be-9baf-75c3c1701d21 None admin] PATH_INFO: `/v3/projects/8d8519390e3f4536964a76bc41b6e43c` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:24.943737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340ab66-8f74-40be-9baf-75c3c1701d21 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8519390e3f4536964a76bc41b6e43c)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:24.945351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7340ab66-8f74-40be-9baf-75c3c1701d21 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:24.947481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 209/415] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:24 2022] GET /identity/v3/projects/8d8519390e3f4536964a76bc41b6e43c => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:24.952933 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340ab66-8f74-40be-9baf-75c3c1701d21 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:24.953123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-73c1fcfb-0c53-44b1-bd03-1648dc46efce None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:24.954246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-73c1fcfb-0c53-44b1-bd03-1648dc46efce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:25.012962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:25.073545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7340ab66-8f74-40be-9baf-75c3c1701d21 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:25.095074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 207/416] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:25.106300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719f90d4-f061-4136-a844-dfe81c56ae0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:25.106959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f90d4-f061-4136-a844-dfe81c56ae0b None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:25.107253 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f90d4-f061-4136-a844-dfe81c56ae0b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:25.107526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719f90d4-f061-4136-a844-dfe81c56ae0b None admin] PATH_INFO: `/v3/projects/8d8519390e3f4536964a76bc41b6e43c/groups/41f40db10ab7437abd9c4505024abf7b/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:25.109491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f90d4-f061-4136-a844-dfe81c56ae0b None admin] RBAC: Authorizing `identity:create_grant(project_id=8d8519390e3f4536964a76bc41b6e43c, group_id=41f40db10ab7437abd9c4505024abf7b, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:25.110920 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719f90d4-f061-4136-a844-dfe81c56ae0b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:25.139958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 210/417] 10.222.0.65 () {58 vars in 1442 bytes} [Wed Aug 31 18:59:24 2022] PUT /identity/v3/projects/8d8519390e3f4536964a76bc41b6e43c/groups/41f40db10ab7437abd9c4505024abf7b/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 18:59:25.245013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e86eb5-9f09-4c61-afb0-a2b933050c15 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:25.245354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e86eb5-9f09-4c61-afb0-a2b933050c15 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:25.245600 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e86eb5-9f09-4c61-afb0-a2b933050c15 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:25.247317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 208/418] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:25.253162 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da654a-de64-42cd-b452-dfce3575ce9c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:25.253823 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da654a-de64-42cd-b452-dfce3575ce9c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:25.254101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da654a-de64-42cd-b452-dfce3575ce9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:25.278908 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-12da654a-de64-42cd-b452-dfce3575ce9c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:25.280455 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-12da654a-de64-42cd-b452-dfce3575ce9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:25.297047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:25.365106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 211/419] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:25.419462 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e86eb5-9f09-4c61-afb0-a2b933050c15 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:25.557826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e86eb5-9f09-4c61-afb0-a2b933050c15 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:25.593405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77816e-b65b-4647-8a59-c90bca1a39dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:25.594033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77816e-b65b-4647-8a59-c90bca1a39dd None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:25.594312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77816e-b65b-4647-8a59-c90bca1a39dd None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:25.594583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba77816e-b65b-4647-8a59-c90bca1a39dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:25.610925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba77816e-b65b-4647-8a59-c90bca1a39dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:25.615858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-ba77816e-b65b-4647-8a59-c90bca1a39dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:25.615858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:25.615858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:25.615858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:25.615858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:25.615858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:25.615858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:25.615858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:25.615858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:25.615858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:25.615858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:25.615858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:25.615858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:25.615858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:25.615858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:25.615858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:25.616711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:25.616711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:25.616711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:25.616711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:25.616711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:25.616711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:25.616711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:25.616711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:25.616711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:25.616711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:25.616711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:25.616711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:25.616711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:25.616711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:25.616711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:25.616711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:25.617501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:25.617501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:25.617501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:25.617501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:25.617501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:25.617501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:25.617501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:25.617501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:25.617501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:25.617501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:25.617501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:25.617501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:25.617501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 209/420] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:25.623770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da654a-de64-42cd-b452-dfce3575ce9c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:25.669768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77816e-b65b-4647-8a59-c90bca1a39dd None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:25.711385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:25.775741 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12da654a-de64-42cd-b452-dfce3575ce9c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:25.801502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba77816e-b65b-4647-8a59-c90bca1a39dd None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:25.811607 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65052037-7c9d-44f9-9fb2-843d9806c316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:25.812141 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65052037-7c9d-44f9-9fb2-843d9806c316 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:25.812420 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65052037-7c9d-44f9-9fb2-843d9806c316 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:25.812689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65052037-7c9d-44f9-9fb2-843d9806c316 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:25.813408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65052037-7c9d-44f9-9fb2-843d9806c316 None admin] RBAC: Adding query filter params (name=member) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:25.813947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65052037-7c9d-44f9-9fb2-843d9806c316 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:25.815155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65052037-7c9d-44f9-9fb2-843d9806c316 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:25.824302 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 212/421] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:25.830434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65052037-7c9d-44f9-9fb2-843d9806c316 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:25.849651 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:25.850410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbdf4f-8031-4262-885e-76f212551aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:25.852150 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbdf4f-8031-4262-885e-76f212551aaf None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:25.852729 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbdf4f-8031-4262-885e-76f212551aaf None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:25.853387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bbdf4f-8031-4262-885e-76f212551aaf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:25.870337 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bbdf4f-8031-4262-885e-76f212551aaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:25.877464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-39bbdf4f-8031-4262-885e-76f212551aaf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:59:25.877464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:25.877464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:25.877464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:25.877464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:25.877464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:25.877464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:25.877464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:25.877464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:25.877464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:25.877464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:25.877464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:25.877464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:25.877464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:25.877464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:25.877464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:25.878311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:25.878311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:25.878311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:25.878311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:25.878311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:25.878311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:25.878311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:25.878311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:25.878311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:25.878311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:25.878311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:25.878311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:25.878311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:25.878311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:25.878311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:25.878311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:25.879109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:25.879109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:25.879109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:25.879109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:25.879109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:25.879109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:25.879109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:25.879109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:25.879109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:25.879109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:25.879109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:59:25.879109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:25.885652 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 210/422] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 18:59:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:25.894092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbdf4f-8031-4262-885e-76f212551aaf None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:26.002612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65052037-7c9d-44f9-9fb2-843d9806c316 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:26.033983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:26.069826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bbdf4f-8031-4262-885e-76f212551aaf None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:26.071698 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7b484-bff0-46c0-b953-2dab31eb2117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:26.072300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7b484-bff0-46c0-b953-2dab31eb2117 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:26.072568 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7b484-bff0-46c0-b953-2dab31eb2117 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:26.072828 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7b484-bff0-46c0-b953-2dab31eb2117 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:26.134485 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d0fc3-206e-429a-abf2-fbe16c2e8add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:26.135084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d0fc3-206e-429a-abf2-fbe16c2e8add None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:26.135355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d0fc3-206e-429a-abf2-fbe16c2e8add None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:26.135620 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d0fc3-206e-429a-abf2-fbe16c2e8add None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:26.136292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d0fc3-206e-429a-abf2-fbe16c2e8add None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:26.136590 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d0fc3-206e-429a-abf2-fbe16c2e8add None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:26.137960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d0fc3-206e-429a-abf2-fbe16c2e8add None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:26.155891 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 211/423] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:26.174602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7b484-bff0-46c0-b953-2dab31eb2117 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:26.176048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7b484-bff0-46c0-b953-2dab31eb2117 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:26.178612 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d0fc3-206e-429a-abf2-fbe16c2e8add None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:26.208709 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:26.305511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 213/424] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 18:59:26.311464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7b484-bff0-46c0-b953-2dab31eb2117 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:26.407223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d0fc3-206e-429a-abf2-fbe16c2e8add None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:26.421941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7b484-bff0-46c0-b953-2dab31eb2117 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:26.436744 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:26.469562 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0ec65e-a2fa-4bfb-95e9-1a874720ce69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:26.470267 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ec65e-a2fa-4bfb-95e9-1a874720ce69 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:26.470595 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ec65e-a2fa-4bfb-95e9-1a874720ce69 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:26.470794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0ec65e-a2fa-4bfb-95e9-1a874720ce69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:26.472280 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c86d6-5ea6-42a1-8b0e-93f17f392b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:26.473131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c86d6-5ea6-42a1-8b0e-93f17f392b18 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:26.473629 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c86d6-5ea6-42a1-8b0e-93f17f392b18 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:26.474092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238c86d6-5ea6-42a1-8b0e-93f17f392b18 None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:26.481322 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c86d6-5ea6-42a1-8b0e-93f17f392b18 None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:26.483047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238c86d6-5ea6-42a1-8b0e-93f17f392b18 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:26.483774 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:26.485469 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 214/425] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:26 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:26.491374 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c86d6-5ea6-42a1-8b0e-93f17f392b18 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:26.547801 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ec65e-a2fa-4bfb-95e9-1a874720ce69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:26.549548 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0ec65e-a2fa-4bfb-95e9-1a874720ce69 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:26.643931 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238c86d6-5ea6-42a1-8b0e-93f17f392b18 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:26.657553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 212/426] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 18:59:26.659018 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a278c59-f941-43b4-8eb7-632b1e9beec0 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:26.659447 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a278c59-f941-43b4-8eb7-632b1e9beec0 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:26.659839 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a278c59-f941-43b4-8eb7-632b1e9beec0 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:26.661522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 213/427] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:26.663122 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a278c59-f941-43b4-8eb7-632b1e9beec0 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:26.680853 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e52671-fd2c-46ab-a7da-8d4e79a00d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:26.681632 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e52671-fd2c-46ab-a7da-8d4e79a00d84 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:26.681943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e52671-fd2c-46ab-a7da-8d4e79a00d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:26.682286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e52671-fd2c-46ab-a7da-8d4e79a00d84 None admin] PATH_INFO: `/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:26.683500 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e52671-fd2c-46ab-a7da-8d4e79a00d84 None admin] RBAC: Authorizing `identity:get_group(group_id=9aa3e6fe034d493bb846ba2e61112d0b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:26.685044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e52671-fd2c-46ab-a7da-8d4e79a00d84 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:26.686787 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 215/428] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 18:59:26 2022] GET /identity/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:26.688132 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a4cac-f9b0-4e85-a504-aaaf546fffdc None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:26.688481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a4cac-f9b0-4e85-a504-aaaf546fffdc None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:26.688826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a4cac-f9b0-4e85-a504-aaaf546fffdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:26.708457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b70a4cac-f9b0-4e85-a504-aaaf546fffdc None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:26.710005 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b70a4cac-f9b0-4e85-a504-aaaf546fffdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:26.711027 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:26.776010 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:26.778908 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a278c59-f941-43b4-8eb7-632b1e9beec0 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:26.809959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 216/429] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:26.811488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a4cac-f9b0-4e85-a504-aaaf546fffdc None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:26.831168 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2683455-70da-4118-a7b8-2fb3bdef2485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:26.831743 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2683455-70da-4118-a7b8-2fb3bdef2485 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:26.831993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2683455-70da-4118-a7b8-2fb3bdef2485 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:26.832257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2683455-70da-4118-a7b8-2fb3bdef2485 None admin] PATH_INFO: `/v3/projects/7d21b4b8971a47ff8985309015eb53d4` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:26.833421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2683455-70da-4118-a7b8-2fb3bdef2485 None admin] RBAC: Authorizing `identity:get_project(project_id=7d21b4b8971a47ff8985309015eb53d4)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:26.834845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2683455-70da-4118-a7b8-2fb3bdef2485 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:26.845702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 214/430] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:26 2022] GET /identity/v3/projects/7d21b4b8971a47ff8985309015eb53d4 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:26.846570 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2683455-70da-4118-a7b8-2fb3bdef2485 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:26.929171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a4cac-f9b0-4e85-a504-aaaf546fffdc None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:26.971040 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a873e0-e0fa-4b47-978f-7e3975045f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:26.972338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a873e0-e0fa-4b47-978f-7e3975045f78 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:26.972852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a873e0-e0fa-4b47-978f-7e3975045f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:26.973354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a873e0-e0fa-4b47-978f-7e3975045f78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:26.974278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a873e0-e0fa-4b47-978f-7e3975045f78 None admin] RBAC: Adding query filter params (group.id=9aa3e6fe034d493bb846ba2e61112d0b, role.id=b80a0d1a8d164b53bef5726b92e68676, scope.project.id=7d21b4b8971a47ff8985309015eb53d4) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:26.974796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a873e0-e0fa-4b47-978f-7e3975045f78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:26.976731 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a873e0-e0fa-4b47-978f-7e3975045f78 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:26.984744 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 217/431] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 18:59:26 2022] GET /identity/v3/role_assignments?group.id=9aa3e6fe034d493bb846ba2e61112d0b&scope.project.id=7d21b4b8971a47ff8985309015eb53d4&role.id=b80a0d1a8d164b53bef5726b92e68676 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:27.029682 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2683455-70da-4118-a7b8-2fb3bdef2485 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:27.041918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:27.068702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0a572-f61f-4951-9fed-49dff60a3423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:27.069830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0a572-f61f-4951-9fed-49dff60a3423 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:27.070423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0a572-f61f-4951-9fed-49dff60a3423 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:27.070900 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e0a572-f61f-4951-9fed-49dff60a3423 None admin] PATH_INFO: `/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:27.072407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0a572-f61f-4951-9fed-49dff60a3423 None admin] RBAC: Authorizing `identity:get_group(group_id=9aa3e6fe034d493bb846ba2e61112d0b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:27.074121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e0a572-f61f-4951-9fed-49dff60a3423 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:27.076270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 215/432] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 18:59:26 2022] GET /identity/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:27.081925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a873e0-e0fa-4b47-978f-7e3975045f78 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:27.100367 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0a572-f61f-4951-9fed-49dff60a3423 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:27.213379 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:27.225884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e0a572-f61f-4951-9fed-49dff60a3423 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:27.267552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc328a00-316b-4a91-ba81-3d7ea93e086c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:27.268828 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc328a00-316b-4a91-ba81-3d7ea93e086c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:27.269503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc328a00-316b-4a91-ba81-3d7ea93e086c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:27.269933 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc328a00-316b-4a91-ba81-3d7ea93e086c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:27.284759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc328a00-316b-4a91-ba81-3d7ea93e086c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:27.294250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-cc328a00-316b-4a91-ba81-3d7ea93e086c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:27.294250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:27.294250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:27.294250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:27.294250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:27.294250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:27.294250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:27.294250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:27.294250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:27.294250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:27.294250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:27.294250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:27.294250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:27.294250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:27.294250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:27.294250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:27.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:27.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:27.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:27.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:27.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:27.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:27.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:27.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:27.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:27.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:27.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:27.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:27.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:27.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:27.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:27.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:27.296157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:27.296157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:27.296157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:27.296157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:27.296157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:27.296157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:27.296157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:27.296157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:27.296157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:27.296157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:27.296157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 31 18:59:27.296157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:27.301210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 216/433] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:27.304855 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc328a00-316b-4a91-ba81-3d7ea93e086c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:27.310131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a873e0-e0fa-4b47-978f-7e3975045f78 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:27.349998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0087a2f-4126-4308-abfc-0a9397e86dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:27.350756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0087a2f-4126-4308-abfc-0a9397e86dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:27.351131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0087a2f-4126-4308-abfc-0a9397e86dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:27.351499 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0087a2f-4126-4308-abfc-0a9397e86dd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:27.352364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0087a2f-4126-4308-abfc-0a9397e86dd9 None admin] RBAC: Adding query filter params (group.id=9aa3e6fe034d493bb846ba2e61112d0b, role.id=470b660efbe04083a803e79d8b647160, scope.project.id=7d21b4b8971a47ff8985309015eb53d4) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:27.352745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0087a2f-4126-4308-abfc-0a9397e86dd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:27.354969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0087a2f-4126-4308-abfc-0a9397e86dd9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:27.357931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:27.365404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 218/434] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 18:59:27 2022] GET /identity/v3/role_assignments?group.id=9aa3e6fe034d493bb846ba2e61112d0b&scope.project.id=7d21b4b8971a47ff8985309015eb53d4&role.id=470b660efbe04083a803e79d8b647160 => generated 611 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:27.436111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc328a00-316b-4a91-ba81-3d7ea93e086c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:27.442773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e3386-24b6-4076-8982-0f0b5743a91e None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:27.443186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e3386-24b6-4076-8982-0f0b5743a91e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:27.443553 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e3386-24b6-4076-8982-0f0b5743a91e None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:27.445383 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 219/435] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 18:59:27.454671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adeb9f4-e5bd-4f45-8038-ffe37def6bb8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:27.454951 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adeb9f4-e5bd-4f45-8038-ffe37def6bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:27.455222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adeb9f4-e5bd-4f45-8038-ffe37def6bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:27.477241 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-5adeb9f4-e5bd-4f45-8038-ffe37def6bb8 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:27.478463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5adeb9f4-e5bd-4f45-8038-ffe37def6bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:27.481046 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b23ca-0df7-429b-a33c-b4e02ede0268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:27.481889 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b23ca-0df7-429b-a33c-b4e02ede0268 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:27.482348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b23ca-0df7-429b-a33c-b4e02ede0268 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:27.482765 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b23ca-0df7-429b-a33c-b4e02ede0268 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:27.483587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b23ca-0df7-429b-a33c-b4e02ede0268 None admin] RBAC: Adding query filter params (name=member) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:27.484033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b23ca-0df7-429b-a33c-b4e02ede0268 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:27.485695 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b23ca-0df7-429b-a33c-b4e02ede0268 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:27.495667 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 217/436] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 18:59:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:27.497380 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d2208-cd7c-4c08-9c41-26b185bd4084 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:27.497811 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d2208-cd7c-4c08-9c41-26b185bd4084 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:27.498241 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7d2208-cd7c-4c08-9c41-26b185bd4084 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:27.500233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 218/437] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:27.503644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d2208-cd7c-4c08-9c41-26b185bd4084 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:27.569235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 220/438] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:27.569235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754f381-0ad8-415d-87ae-dbf9c9185b66 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:27.569235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754f381-0ad8-415d-87ae-dbf9c9185b66 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:27.569235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5754f381-0ad8-415d-87ae-dbf9c9185b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:27.590929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-5754f381-0ad8-415d-87ae-dbf9c9185b66 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:27.592481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5754f381-0ad8-415d-87ae-dbf9c9185b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:27.627367 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:27.640393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:27.642440 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7d2208-cd7c-4c08-9c41-26b185bd4084 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:27.677165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 221/439] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:27.680053 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbcab9-0e4f-4ab4-bf89-ab02f6c5b3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:27.680664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbcab9-0e4f-4ab4-bf89-ab02f6c5b3ea None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:27.680939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbcab9-0e4f-4ab4-bf89-ab02f6c5b3ea None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:27.681223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bbcab9-0e4f-4ab4-bf89-ab02f6c5b3ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:27.755675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbcab9-0e4f-4ab4-bf89-ab02f6c5b3ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:27.757495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bbcab9-0e4f-4ab4-bf89-ab02f6c5b3ea None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:27.850419 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754f381-0ad8-415d-87ae-dbf9c9185b66 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:27.862845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:27.869259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 219/440] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:27.874295 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbcab9-0e4f-4ab4-bf89-ab02f6c5b3ea None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:27.991727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bbcab9-0e4f-4ab4-bf89-ab02f6c5b3ea None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:27.999047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5754f381-0ad8-415d-87ae-dbf9c9185b66 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:28.027168 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ced6a-83f7-44f1-83a3-17050bbbc4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:28.027786 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ced6a-83f7-44f1-83a3-17050bbbc4e1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:28.028075 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ced6a-83f7-44f1-83a3-17050bbbc4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:28.028346 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3ced6a-83f7-44f1-83a3-17050bbbc4e1 None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:28.029485 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ced6a-83f7-44f1-83a3-17050bbbc4e1 None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:28.030915 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3ced6a-83f7-44f1-83a3-17050bbbc4e1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:28.036944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 220/441] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:27 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:28.037079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01942dc7-fde4-4daa-8461-7849253c4466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:28.037202 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ced6a-83f7-44f1-83a3-17050bbbc4e1 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:28.037309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01942dc7-fde4-4daa-8461-7849253c4466 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:28.037309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01942dc7-fde4-4daa-8461-7849253c4466 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:28.037309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01942dc7-fde4-4daa-8461-7849253c4466 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:28.044589 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01942dc7-fde4-4daa-8461-7849253c4466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:28.049431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-01942dc7-fde4-4daa-8461-7849253c4466 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:59:28.049431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:28.049431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:28.049431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:28.049431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:28.049431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:28.049431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:28.049431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:28.049431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:28.049431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:28.049431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:28.049431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:28.049431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:28.049431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:28.049431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:28.049431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:28.050323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:28.050323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:28.050323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:28.050323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:28.050323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:28.050323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:28.050323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:28.050323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:28.050323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:28.050323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:28.050323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:28.050323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:28.050323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:28.050323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:28.050323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:28.050323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:28.051108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:28.051108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:28.051108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:28.051108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:28.051108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:28.051108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:28.051108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:28.051108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:28.051108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:28.051108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:28.051108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:59:28.051108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:28.051108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 222/442] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:59:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:28.051773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01942dc7-fde4-4daa-8461-7849253c4466 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:28.078247 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:28.104026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:28.137332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3ced6a-83f7-44f1-83a3-17050bbbc4e1 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:28.151018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01942dc7-fde4-4daa-8461-7849253c4466 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:28.169882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a3e39-4811-45b3-8c2a-9535625b13f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:28.170496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a3e39-4811-45b3-8c2a-9535625b13f1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:28.170777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a3e39-4811-45b3-8c2a-9535625b13f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:28.171045 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a3e39-4811-45b3-8c2a-9535625b13f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:28.180752 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a3e39-4811-45b3-8c2a-9535625b13f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:28.184530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcd898-a633-4553-b7ef-5e370e70c4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:28.185140 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcd898-a633-4553-b7ef-5e370e70c4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:28.185445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcd898-a633-4553-b7ef-5e370e70c4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:28.185717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bcd898-a633-4553-b7ef-5e370e70c4f2 None admin] PATH_INFO: `/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:28.186116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-054a3e39-4811-45b3-8c2a-9535625b13f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:59:28.186116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:28.186116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:28.186116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:28.186116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:28.186116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:28.186116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:28.186116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:28.186116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:28.186116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:28.186116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:28.186116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:28.186116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:28.186116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:28.186116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:28.186116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:28.186996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:28.186996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:28.186996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:28.186996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:28.186996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:28.186996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:28.186996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:28.186996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:28.186996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:28.186996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:28.186996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:28.186996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:28.186996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:28.186996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:28.186996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:28.186996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:28.187926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:28.187926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:28.187926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:28.187926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:28.187926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:28.187926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:28.187926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:28.187926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:28.187926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:28.187926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:28.187926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:59:28.187926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:28.188763 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bcd898-a633-4553-b7ef-5e370e70c4f2 None admin] RBAC: Authorizing `identity:get_group(group_id=9aa3e6fe034d493bb846ba2e61112d0b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:28.188831 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 221/443] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 18:59:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:28.188894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bcd898-a633-4553-b7ef-5e370e70c4f2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:28.189899 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 223/444] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 18:59:28 2022] GET /identity/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:28.190568 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a3e39-4811-45b3-8c2a-9535625b13f1 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:28.192822 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcd898-a633-4553-b7ef-5e370e70c4f2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:28.288880 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:28.291488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a3e39-4811-45b3-8c2a-9535625b13f1 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:28.296686 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bcd898-a633-4553-b7ef-5e370e70c4f2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:28.318122 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:28.326687 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445d61b-0a19-435e-9395-564e2d5f1dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:28.327290 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445d61b-0a19-435e-9395-564e2d5f1dfa None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:28.327568 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445d61b-0a19-435e-9395-564e2d5f1dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:28.327866 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7445d61b-0a19-435e-9395-564e2d5f1dfa None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:28.328526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445d61b-0a19-435e-9395-564e2d5f1dfa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:28.328827 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445d61b-0a19-435e-9395-564e2d5f1dfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:28.330237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7445d61b-0a19-435e-9395-564e2d5f1dfa None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:28.331485 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73109429-58ff-4975-8c7e-ffe8e1b08408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:28.332077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73109429-58ff-4975-8c7e-ffe8e1b08408 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:28.332360 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73109429-58ff-4975-8c7e-ffe8e1b08408 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:28.332628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73109429-58ff-4975-8c7e-ffe8e1b08408 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:28.333304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73109429-58ff-4975-8c7e-ffe8e1b08408 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:28.333601 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73109429-58ff-4975-8c7e-ffe8e1b08408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:28.334929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73109429-58ff-4975-8c7e-ffe8e1b08408 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:28.338314 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 222/445] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:28.339092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445d61b-0a19-435e-9395-564e2d5f1dfa None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:28.350687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 224/446] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:28.350687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73109429-58ff-4975-8c7e-ffe8e1b08408 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:28.446737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73109429-58ff-4975-8c7e-ffe8e1b08408 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:28.448921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7445d61b-0a19-435e-9395-564e2d5f1dfa None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:28.479786 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfed6c-405e-48d1-a656-d6216c68afb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:28.480927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfed6c-405e-48d1-a656-d6216c68afb9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:28.481230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfed6c-405e-48d1-a656-d6216c68afb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:28.481505 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadfed6c-405e-48d1-a656-d6216c68afb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:28.482243 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715b62f-136e-4ec3-a8d0-fd407a431490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:28.482835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715b62f-136e-4ec3-a8d0-fd407a431490 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:28.483114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715b62f-136e-4ec3-a8d0-fd407a431490 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:28.483386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9715b62f-136e-4ec3-a8d0-fd407a431490 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:28.484108 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715b62f-136e-4ec3-a8d0-fd407a431490 None admin] RBAC: Adding query filter params (group.id=9aa3e6fe034d493bb846ba2e61112d0b, role.id=b80a0d1a8d164b53bef5726b92e68676, scope.project.id=8ab3b89f1290419e93f22ea7eee77da8) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:28.484413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715b62f-136e-4ec3-a8d0-fd407a431490 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:28.485873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9715b62f-136e-4ec3-a8d0-fd407a431490 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:28.491775 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 223/447] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 18:59:28 2022] GET /identity/v3/role_assignments?group.id=9aa3e6fe034d493bb846ba2e61112d0b&scope.project.id=8ab3b89f1290419e93f22ea7eee77da8&role.id=b80a0d1a8d164b53bef5726b92e68676 => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:28.492547 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715b62f-136e-4ec3-a8d0-fd407a431490 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:28.518020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:28.558333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:28.572943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadfed6c-405e-48d1-a656-d6216c68afb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:28.574490 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadfed6c-405e-48d1-a656-d6216c68afb9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:28.610712 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9715b62f-136e-4ec3-a8d0-fd407a431490 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:28.653733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c186aa-c349-4090-bb66-dd729b2309ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:28.654457 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c186aa-c349-4090-bb66-dd729b2309ac None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:28.654743 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c186aa-c349-4090-bb66-dd729b2309ac None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:28.655013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c186aa-c349-4090-bb66-dd729b2309ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:28.689968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 225/448] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:28.696163 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfed6c-405e-48d1-a656-d6216c68afb9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:28.735431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c186aa-c349-4090-bb66-dd729b2309ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:28.737323 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c186aa-c349-4090-bb66-dd729b2309ac None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:28.759502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:28.801264 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:28.808245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadfed6c-405e-48d1-a656-d6216c68afb9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:28.850357 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69af82-c29d-428e-b8fb-7a4408798985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:28.850907 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69af82-c29d-428e-b8fb-7a4408798985 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:28.851182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69af82-c29d-428e-b8fb-7a4408798985 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:28.851446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf69af82-c29d-428e-b8fb-7a4408798985 None admin] PATH_INFO: `/v3/projects/8d8519390e3f4536964a76bc41b6e43c` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:28.852568 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69af82-c29d-428e-b8fb-7a4408798985 None admin] RBAC: Authorizing `identity:get_project(project_id=8d8519390e3f4536964a76bc41b6e43c)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:28.854020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf69af82-c29d-428e-b8fb-7a4408798985 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:28.855677 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 226/449] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:28 2022] GET /identity/v3/projects/8d8519390e3f4536964a76bc41b6e43c => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:28.860328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 224/450] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:28.861135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69af82-c29d-428e-b8fb-7a4408798985 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:28.865601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c186aa-c349-4090-bb66-dd729b2309ac None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:28.964040 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf69af82-c29d-428e-b8fb-7a4408798985 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:28.965938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c186aa-c349-4090-bb66-dd729b2309ac None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:28.987924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:28.999199 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8f130-e438-459c-8299-013827035c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:28.999375 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698872b3-b14a-4d7e-b99a-a5a132e4d0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:28.999854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8f130-e438-459c-8299-013827035c6b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:28.999961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698872b3-b14a-4d7e-b99a-a5a132e4d0e8 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:29.000126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698872b3-b14a-4d7e-b99a-a5a132e4d0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:29.000219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8f130-e438-459c-8299-013827035c6b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:29.000395 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb8f130-e438-459c-8299-013827035c6b None admin] PATH_INFO: `/v3/groups/41f40db10ab7437abd9c4505024abf7b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:29.000491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698872b3-b14a-4d7e-b99a-a5a132e4d0e8 None admin] PATH_INFO: `/v3/projects/8ab3b89f1290419e93f22ea7eee77da8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:29.001524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698872b3-b14a-4d7e-b99a-a5a132e4d0e8 None admin] RBAC: Authorizing `identity:get_project(project_id=8ab3b89f1290419e93f22ea7eee77da8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:29.001621 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb8f130-e438-459c-8299-013827035c6b None admin] RBAC: Authorizing `identity:get_group(group_id=41f40db10ab7437abd9c4505024abf7b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:29.002948 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698872b3-b14a-4d7e-b99a-a5a132e4d0e8 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:29.003033 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb8f130-e438-459c-8299-013827035c6b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:29.004726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 227/451] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 18:59:28 2022] GET /identity/v3/groups/41f40db10ab7437abd9c4505024abf7b => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:29.004833 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 225/452] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 18:59:28 2022] GET /identity/v3/projects/8ab3b89f1290419e93f22ea7eee77da8 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:29.010055 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8f130-e438-459c-8299-013827035c6b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:29.010169 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698872b3-b14a-4d7e-b99a-a5a132e4d0e8 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:29.035800 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:29.107949 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb8f130-e438-459c-8299-013827035c6b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:29.108700 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698872b3-b14a-4d7e-b99a-a5a132e4d0e8 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:29.142220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e901d6c-0764-4b99-bbd7-b2d6827016a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:29.142389 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0f567-ab95-4770-a9f8-5aa07fcefbbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:29.142835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e901d6c-0764-4b99-bbd7-b2d6827016a8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:29.142958 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0f567-ab95-4770-a9f8-5aa07fcefbbf None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:29.143109 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e901d6c-0764-4b99-bbd7-b2d6827016a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:29.143234 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0f567-ab95-4770-a9f8-5aa07fcefbbf None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:29.143376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e901d6c-0764-4b99-bbd7-b2d6827016a8 None admin] PATH_INFO: `/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:29.143499 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0f567-ab95-4770-a9f8-5aa07fcefbbf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:29.144215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0f567-ab95-4770-a9f8-5aa07fcefbbf None admin] RBAC: Adding query filter params (group.id=41f40db10ab7437abd9c4505024abf7b, role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=8d8519390e3f4536964a76bc41b6e43c) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:29.144466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e901d6c-0764-4b99-bbd7-b2d6827016a8 None admin] RBAC: Authorizing `identity:get_group(group_id=9aa3e6fe034d493bb846ba2e61112d0b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:29.144556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0f567-ab95-4770-a9f8-5aa07fcefbbf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:29.145918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e901d6c-0764-4b99-bbd7-b2d6827016a8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:29.146032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0f567-ab95-4770-a9f8-5aa07fcefbbf None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:29.147530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 228/453] 10.222.0.65 () {56 vars in 1180 bytes} [Wed Aug 31 18:59:29 2022] GET /identity/v3/groups/9aa3e6fe034d493bb846ba2e61112d0b => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:29.153759 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e901d6c-0764-4b99-bbd7-b2d6827016a8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:29.153998 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 226/454] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 18:59:29 2022] GET /identity/v3/role_assignments?group.id=41f40db10ab7437abd9c4505024abf7b&scope.project.id=8d8519390e3f4536964a76bc41b6e43c&role.id=3580c894dd974d35bc95848b8ba22598 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:29.223210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:29.265498 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e901d6c-0764-4b99-bbd7-b2d6827016a8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:29.300085 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0886f3-7bbb-4280-8b65-2a7d2b3b2a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:29.300705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0886f3-7bbb-4280-8b65-2a7d2b3b2a61 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:29.300936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0886f3-7bbb-4280-8b65-2a7d2b3b2a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:29.301210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0886f3-7bbb-4280-8b65-2a7d2b3b2a61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:29.301920 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0886f3-7bbb-4280-8b65-2a7d2b3b2a61 None admin] RBAC: Adding query filter params (group.id=9aa3e6fe034d493bb846ba2e61112d0b, role.id=470b660efbe04083a803e79d8b647160, scope.project.id=8ab3b89f1290419e93f22ea7eee77da8) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:29.302215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0886f3-7bbb-4280-8b65-2a7d2b3b2a61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:29.303618 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0886f3-7bbb-4280-8b65-2a7d2b3b2a61 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:29.309817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 229/455] 10.222.0.65 () {56 vars in 1376 bytes} [Wed Aug 31 18:59:29 2022] GET /identity/v3/role_assignments?group.id=9aa3e6fe034d493bb846ba2e61112d0b&scope.project.id=8ab3b89f1290419e93f22ea7eee77da8&role.id=470b660efbe04083a803e79d8b647160 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:31.779607 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a56083-f457-4b86-8080-c1965dfe16d1 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:31.780656 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a56083-f457-4b86-8080-c1965dfe16d1 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:31.781132 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a56083-f457-4b86-8080-c1965dfe16d1 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:31.782939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 227/456] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:31.788543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ada2f-ab8a-47f2-af07-ab66562a2774 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:31.789378 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ada2f-ab8a-47f2-af07-ab66562a2774 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:31.789867 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65ada2f-ab8a-47f2-af07-ab66562a2774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:31.816551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a65ada2f-ab8a-47f2-af07-ab66562a2774 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:31.818104 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a65ada2f-ab8a-47f2-af07-ab66562a2774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:31.874982 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f4fc2-8332-4522-8a09-7aa86543f7f2 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:31.875610 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f4fc2-8332-4522-8a09-7aa86543f7f2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:31.876112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f4fc2-8332-4522-8a09-7aa86543f7f2 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:31.877880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 228/457] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:31.883125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8311f-c6e9-4235-b1b8-1b78924f3721 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:31.883646 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8311f-c6e9-4235-b1b8-1b78924f3721 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:31.884117 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d8311f-c6e9-4235-b1b8-1b78924f3721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:31.913028 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 230/458] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:59:31.927849 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-33d8311f-c6e9-4235-b1b8-1b78924f3721 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:31.929103 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-33d8311f-c6e9-4235-b1b8-1b78924f3721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:32.066736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 229/459] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:59:32.207354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ada2f-ab8a-47f2-af07-ab66562a2774 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:32.266969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:32.321931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac54d9-7d2b-4bab-96e0-878d017f786b None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:32.322539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac54d9-7d2b-4bab-96e0-878d017f786b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:32.323010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac54d9-7d2b-4bab-96e0-878d017f786b None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:32.325340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 230/460] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:32.329212 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65ada2f-ab8a-47f2-af07-ab66562a2774 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:32.330793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6ac95-a6a3-4709-b3c6-101e6ff01d58 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:32.331285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6ac95-a6a3-4709-b3c6-101e6ff01d58 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:32.331753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d6ac95-a6a3-4709-b3c6-101e6ff01d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:32.369276 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24af21-263f-4b5e-a449-3bd95fec5ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:32.369783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24af21-263f-4b5e-a449-3bd95fec5ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:32.370232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24af21-263f-4b5e-a449-3bd95fec5ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:32.371301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b24af21-263f-4b5e-a449-3bd95fec5ad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:32.387340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24af21-263f-4b5e-a449-3bd95fec5ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:32.389249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d5d6ac95-a6a3-4709-b3c6-101e6ff01d58 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:32.389775 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b24af21-263f-4b5e-a449-3bd95fec5ad8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:32.390644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d5d6ac95-a6a3-4709-b3c6-101e6ff01d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:32.409175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-3b24af21-263f-4b5e-a449-3bd95fec5ad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:32.409175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:32.409175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:32.409175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:32.409175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:32.409175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:32.409175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:32.409175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:32.409175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:32.409175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:32.409175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:32.409175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:32.409175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:32.409175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:32.409175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:32.409175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:32.410297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:32.412205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:32.412205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:32.412205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:32.417529 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:32.420452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 231/461] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:32.424389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24af21-263f-4b5e-a449-3bd95fec5ad8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:32.526887 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b24af21-263f-4b5e-a449-3bd95fec5ad8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:32.534803 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:32.549569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 231/462] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:59:32.551300 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6ac95-a6a3-4709-b3c6-101e6ff01d58 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:32.570809 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8909c7-5656-44cc-b9d7-e855991955c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:32.571417 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8909c7-5656-44cc-b9d7-e855991955c9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:32.571694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8909c7-5656-44cc-b9d7-e855991955c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:32.572018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8909c7-5656-44cc-b9d7-e855991955c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:32.572607 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8909c7-5656-44cc-b9d7-e855991955c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:32.572906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8909c7-5656-44cc-b9d7-e855991955c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:32.574947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8909c7-5656-44cc-b9d7-e855991955c9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:32.596731 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 232/463] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:32.603919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8909c7-5656-44cc-b9d7-e855991955c9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:32.732938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d6ac95-a6a3-4709-b3c6-101e6ff01d58 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:32.767789 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8909c7-5656-44cc-b9d7-e855991955c9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:32.770238 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b94db8-9009-4fbc-b7c2-43c3ec8caf08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:32.771079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b94db8-9009-4fbc-b7c2-43c3ec8caf08 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:32.771498 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b94db8-9009-4fbc-b7c2-43c3ec8caf08 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:32.772010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b94db8-9009-4fbc-b7c2-43c3ec8caf08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:32.784473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b94db8-9009-4fbc-b7c2-43c3ec8caf08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:32.786572 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b94db8-9009-4fbc-b7c2-43c3ec8caf08 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:32.794734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:32.804340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-59b94db8-9009-4fbc-b7c2-43c3ec8caf08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:32.804340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:32.804340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:32.804340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:32.804340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:32.804340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:32.804340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:32.804340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:32.804340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:32.804340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:32.804340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:32.804340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:32.804340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:32.804340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:32.804340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:32.804340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:32.805378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:32.806226 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:32.806226 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:32.806226 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:32.806226 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 232/464] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:32.809407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3771d-8299-443b-9a26-aefb8be31378 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:32.809683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3771d-8299-443b-9a26-aefb8be31378 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:32.809949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a3771d-8299-443b-9a26-aefb8be31378 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:32.811514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 233/465] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:32.812480 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db52b05-fe98-4e3c-983f-dc22ff71ba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:32.813637 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db52b05-fe98-4e3c-983f-dc22ff71ba7c None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:32.814084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db52b05-fe98-4e3c-983f-dc22ff71ba7c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:32.814517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db52b05-fe98-4e3c-983f-dc22ff71ba7c None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:32.815461 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db52b05-fe98-4e3c-983f-dc22ff71ba7c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:32.816809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52decc47-7b74-49c5-8403-9123877c6647 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:32.817083 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52decc47-7b74-49c5-8403-9123877c6647 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:32.817370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52decc47-7b74-49c5-8403-9123877c6647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:32.821382 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db52b05-fe98-4e3c-983f-dc22ff71ba7c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:32.838248 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-52decc47-7b74-49c5-8403-9123877c6647 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:32.839671 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-52decc47-7b74-49c5-8403-9123877c6647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:32.853895 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 233/466] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 18:59:32 2022] POST /identity/v3/users => generated 246 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:59:32.855657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db52b05-fe98-4e3c-983f-dc22ff71ba7c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:32.891655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:32.940738 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 234/467] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:32.942345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52decc47-7b74-49c5-8403-9123877c6647 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:33.018004 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db52b05-fe98-4e3c-983f-dc22ff71ba7c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:33.062492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801db26a-d52e-4b27-9456-0b2f27d6b2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:33.063111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801db26a-d52e-4b27-9456-0b2f27d6b2ae None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:33.063389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801db26a-d52e-4b27-9456-0b2f27d6b2ae None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:33.063669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801db26a-d52e-4b27-9456-0b2f27d6b2ae None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:33.064429 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801db26a-d52e-4b27-9456-0b2f27d6b2ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:33.064734 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801db26a-d52e-4b27-9456-0b2f27d6b2ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:33.067142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801db26a-d52e-4b27-9456-0b2f27d6b2ae None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:33.068499 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52decc47-7b74-49c5-8403-9123877c6647 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:33.081213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 234/468] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:33.086446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801db26a-d52e-4b27-9456-0b2f27d6b2ae None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:33.102444 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:33.113120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8e649-84df-479c-88ac-4ec3c5911afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:33.114108 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8e649-84df-479c-88ac-4ec3c5911afe None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:33.114607 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8e649-84df-479c-88ac-4ec3c5911afe None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:33.115072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e8e649-84df-479c-88ac-4ec3c5911afe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:33.128911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8e649-84df-479c-88ac-4ec3c5911afe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:33.130686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e8e649-84df-479c-88ac-4ec3c5911afe None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:33.154967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-f9e8e649-84df-479c-88ac-4ec3c5911afe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:33.154967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:33.154967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:33.154967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:33.154967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:33.154967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:33.154967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:33.154967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:33.154967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:33.154967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:33.154967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:33.154967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:33.154967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:33.154967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:33.154967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:33.154967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:33.155882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:33.156710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:33.156710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:33.156710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:33.156710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 235/469] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:33.166104 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8e649-84df-479c-88ac-4ec3c5911afe None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:33.203754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:33.237629 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801db26a-d52e-4b27-9456-0b2f27d6b2ae None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:33.278530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81da09d2-3edc-42ab-90f9-ef8744beebad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:33.279136 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da09d2-3edc-42ab-90f9-ef8744beebad None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:33.279409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da09d2-3edc-42ab-90f9-ef8744beebad None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:33.279673 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81da09d2-3edc-42ab-90f9-ef8744beebad None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:33.280317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da09d2-3edc-42ab-90f9-ef8744beebad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:33.281809 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81da09d2-3edc-42ab-90f9-ef8744beebad None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:33.304327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 235/470] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 18:59:33 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:59:33.305685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee9cbb-effd-4247-9057-011fce1e5484 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:33.305964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee9cbb-effd-4247-9057-011fce1e5484 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:33.306234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee9cbb-effd-4247-9057-011fce1e5484 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:33.306691 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e8e649-84df-479c-88ac-4ec3c5911afe None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:33.307492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 236/471] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:33.312326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee9cbb-effd-4247-9057-011fce1e5484 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:33.356866 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e405ce96-4983-498a-b0d2-e9ed2bcf97c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:33.357967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405ce96-4983-498a-b0d2-e9ed2bcf97c4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:33.358604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405ce96-4983-498a-b0d2-e9ed2bcf97c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:33.359206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e405ce96-4983-498a-b0d2-e9ed2bcf97c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:33.360142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405ce96-4983-498a-b0d2-e9ed2bcf97c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:33.360706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405ce96-4983-498a-b0d2-e9ed2bcf97c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:33.362559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e405ce96-4983-498a-b0d2-e9ed2bcf97c4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:33.380648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 236/472] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:33.382421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66899ebf-b8b0-442b-8d92-d2a9f52f1b0c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:33.382955 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66899ebf-b8b0-442b-8d92-d2a9f52f1b0c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:33.383472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66899ebf-b8b0-442b-8d92-d2a9f52f1b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:33.405343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-66899ebf-b8b0-442b-8d92-d2a9f52f1b0c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:33.407165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-66899ebf-b8b0-442b-8d92-d2a9f52f1b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:33.456590 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:33.461042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee9cbb-effd-4247-9057-011fce1e5484 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:33.500660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 237/473] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:33.501343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66899ebf-b8b0-442b-8d92-d2a9f52f1b0c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:33.504020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993193b-ea0e-4329-8cdf-21df4fa78f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:33.504648 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993193b-ea0e-4329-8cdf-21df4fa78f20 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:33.504921 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993193b-ea0e-4329-8cdf-21df4fa78f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:33.505263 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2993193b-ea0e-4329-8cdf-21df4fa78f20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:33.513137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:33.518225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993193b-ea0e-4329-8cdf-21df4fa78f20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:33.519678 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2993193b-ea0e-4329-8cdf-21df4fa78f20 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:33.536534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-2993193b-ea0e-4329-8cdf-21df4fa78f20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:33.536534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:33.536534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:33.536534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:33.536534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:33.536534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:33.536534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:33.536534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:33.536534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:33.536534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:33.536534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:33.536534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:33.536534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:33.536534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:33.536534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:33.536534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:33.537727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:33.538763 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:33.538763 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:33.538763 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:33.541632 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 237/474] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:33.547550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993193b-ea0e-4329-8cdf-21df4fa78f20 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:33.644904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66899ebf-b8b0-442b-8d92-d2a9f52f1b0c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:33.680609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3dffe-26bc-4d18-8d7f-b9ba60a4d015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:33.681353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3dffe-26bc-4d18-8d7f-b9ba60a4d015 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:33.681701 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3dffe-26bc-4d18-8d7f-b9ba60a4d015 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:33.682039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3dffe-26bc-4d18-8d7f-b9ba60a4d015 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:33.682520 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2993193b-ea0e-4329-8cdf-21df4fa78f20 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:33.682802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3dffe-26bc-4d18-8d7f-b9ba60a4d015 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:33.684335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e3dffe-26bc-4d18-8d7f-b9ba60a4d015 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:33.708722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 238/475] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 18:59:33 2022] POST /identity/v3/users => generated 248 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:59:33.722569 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13346fa-9825-449c-9c8d-a100fe6488ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:33.723505 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13346fa-9825-449c-9c8d-a100fe6488ec None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:33.723957 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13346fa-9825-449c-9c8d-a100fe6488ec None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:33.724401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13346fa-9825-449c-9c8d-a100fe6488ec None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:33.725279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13346fa-9825-449c-9c8d-a100fe6488ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:33.725789 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13346fa-9825-449c-9c8d-a100fe6488ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:33.727433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13346fa-9825-449c-9c8d-a100fe6488ec None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:33.742545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 238/476] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:33.748832 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3dffe-26bc-4d18-8d7f-b9ba60a4d015 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:33.785888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:33.865705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3dffe-26bc-4d18-8d7f-b9ba60a4d015 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:33.903555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01c185-efc9-4abd-ab8d-152940b94381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:33.904654 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01c185-efc9-4abd-ab8d-152940b94381 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:33.904943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01c185-efc9-4abd-ab8d-152940b94381 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:33.905236 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c01c185-efc9-4abd-ab8d-152940b94381 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:33.905908 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01c185-efc9-4abd-ab8d-152940b94381 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:33.906176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13346fa-9825-449c-9c8d-a100fe6488ec None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:33.909988 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c01c185-efc9-4abd-ab8d-152940b94381 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:33.932967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 239/477] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 18:59:33 2022] POST /identity/v3/users => generated 249 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:59:33.994790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:34.054515 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13346fa-9825-449c-9c8d-a100fe6488ec None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:34.094110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36865be-f513-496f-92b1-4ecb58619c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:34.094720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36865be-f513-496f-92b1-4ecb58619c3f None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:34.095003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36865be-f513-496f-92b1-4ecb58619c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:34.095270 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36865be-f513-496f-92b1-4ecb58619c3f None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:34.095953 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36865be-f513-496f-92b1-4ecb58619c3f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:34.097338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36865be-f513-496f-92b1-4ecb58619c3f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:34.106075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.sql.core [None req-d36865be-f513-496f-92b1-4ecb58619c3f None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 18:59:34.106075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 18:59:34.106075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [parameters: {'id': '1badbe86a23d4662bf54aec46988dfce', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 18:59:34.106075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81627) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 18:59:34.123993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-d36865be-f513-496f-92b1-4ecb58619c3f None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 18:59:34.123993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:34.123993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 18:59:34.123993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 18:59:34.123993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 18:59:34.123993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 18:59:34.123993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 18:59:34.123993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 18:59:34.123993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 18:59:34.123993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application conn.query(q) Aug 31 18:59:34.123993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 18:59:34.123993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 18:59:34.123993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 18:59:34.123993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application result.read() Aug 31 18:59:34.123993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return ref.to_dict() Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 18:59:34.125098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application next(self.gen) Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application yield resource Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application next(self.gen) Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise self.value Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application session.commit() Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 18:59:34.126234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.session.flush() Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application compat.raise_( Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 18:59:34.127290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 18:59:34.128151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 18:59:34.128151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 18:59:34.128151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 18:59:34.128151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 18:59:34.128151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 18:59:34.128151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 18:59:34.128151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 18:59:34.128151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 18:59:34.128151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 18:59:34.128151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 18:59:34.128151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 18:59:34.128151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 18:59:34.128151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 18:59:34.128151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 18:59:34.128151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 18:59:34.128151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception Aug 31 18:59:34.129056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 18:59:34.129056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 18:59:34.129056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 18:59:34.129056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 18:59:34.129056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 18:59:34.129056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 18:59:34.129056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 18:59:34.129056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application conn.query(q) Aug 31 18:59:34.129056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 18:59:34.129056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 18:59:34.129056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 18:59:34.129056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application result.read() Aug 31 18:59:34.129056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 18:59:34.129056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 18:59:34.129056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 18:59:34.129056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 18:59:34.129893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 18:59:34.129893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 18:59:34.129893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 18:59:34.129893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 18:59:34.129893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 31 18:59:34.129893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 31 18:59:34.129893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [parameters: {'id': '1badbe86a23d4662bf54aec46988dfce', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 31 18:59:34.129893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 18:59:34.129893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:34.129893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 18:59:34.129893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:34.129893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:34.129893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:34.129893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:34.129893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:34.129893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:34.133211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:34.133211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:34.133211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:34.133211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:34.133211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:34.133211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:34.133211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 31 18:59:34.133211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 31 18:59:34.133211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:34.133211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:34.133211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 31 18:59:34.133211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 31 18:59:34.133211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 18:59:34.133211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 18:59:34.133211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 31 18:59:34.133211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:34.135280 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 239/478] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 31 18:59:33 2022] POST /identity/v3/roles => generated 144 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 18:59:34.135415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01c185-efc9-4abd-ab8d-152940b94381 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:34.232793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:34.260906 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c01c185-efc9-4abd-ab8d-152940b94381 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:34.316279 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b624b60-c248-47d1-b4f7-77940d4baf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:34.316984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b624b60-c248-47d1-b4f7-77940d4baf6d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:34.317287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b624b60-c248-47d1-b4f7-77940d4baf6d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:34.317575 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b624b60-c248-47d1-b4f7-77940d4baf6d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:34.334528 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b624b60-c248-47d1-b4f7-77940d4baf6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:34.340437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-9b624b60-c248-47d1-b4f7-77940d4baf6d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:59:34.340437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:34.340437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:34.340437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:34.340437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:34.340437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:34.340437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:34.340437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:34.340437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:34.340437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:34.340437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:34.340437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:34.340437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:34.340437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:34.340437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:34.340437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:34.341332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:34.341332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:34.341332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:34.341332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:34.341332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:34.341332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:34.341332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:34.341332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:34.341332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:34.341332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:34.341332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:34.341332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:34.341332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:34.341332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:34.341332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:34.341332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:34.342126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:34.342126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:34.342126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:34.342126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:34.342126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:34.342126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:34.342126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:34.342126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:34.342126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:34.342126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:34.342126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 31 18:59:34.342126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:34.342126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 240/479] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 18:59:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:34.349100 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36865be-f513-496f-92b1-4ecb58619c3f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:34.506357 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36865be-f513-496f-92b1-4ecb58619c3f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:34.525299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:34.549441 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c40b6c-9639-4d6d-abd5-a2e18d6c9fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:34.550098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c40b6c-9639-4d6d-abd5-a2e18d6c9fe8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:34.550380 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c40b6c-9639-4d6d-abd5-a2e18d6c9fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:34.550649 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c40b6c-9639-4d6d-abd5-a2e18d6c9fe8 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:34.551370 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c40b6c-9639-4d6d-abd5-a2e18d6c9fe8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:34.551673 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c40b6c-9639-4d6d-abd5-a2e18d6c9fe8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:34.553037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c40b6c-9639-4d6d-abd5-a2e18d6c9fe8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:34.564800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 240/480] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:35.153603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a7e92-d7e4-45da-b0b3-7fd7d193daf8 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:35.154149 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a7e92-d7e4-45da-b0b3-7fd7d193daf8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:35.154149 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a7e92-d7e4-45da-b0b3-7fd7d193daf8 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:35.155714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 241/481] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:35.161089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415d5eb-9e4f-4a90-b334-a20e38de43e1 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:35.161546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415d5eb-9e4f-4a90-b334-a20e38de43e1 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:35.161734 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415d5eb-9e4f-4a90-b334-a20e38de43e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:35.192286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f415d5eb-9e4f-4a90-b334-a20e38de43e1 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:35.192286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f415d5eb-9e4f-4a90-b334-a20e38de43e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:35.288194 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 241/482] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:59:35.611187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a7e92-d7e4-45da-b0b3-7fd7d193daf8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:35.717812 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a7e92-d7e4-45da-b0b3-7fd7d193daf8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:35.752816 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34834f64-cb20-4f83-b906-697555d51e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:35.753469 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34834f64-cb20-4f83-b906-697555d51e5c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:35.753746 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34834f64-cb20-4f83-b906-697555d51e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:35.754015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34834f64-cb20-4f83-b906-697555d51e5c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:35.765717 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34834f64-cb20-4f83-b906-697555d51e5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:35.770545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-34834f64-cb20-4f83-b906-697555d51e5c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:35.770545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:35.770545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:35.770545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:35.770545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:35.770545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:35.770545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:35.770545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:35.770545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:35.770545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:35.770545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:35.770545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:35.770545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:35.770545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:35.770545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:35.770545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:35.771452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:35.771452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:35.771452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:35.771452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:35.771452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:35.771452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:35.771452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:35.771452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:35.771452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:35.771452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:35.771452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:35.771452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:35.771452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:35.771452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:35.771452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:35.771452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:35.772224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:35.772224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:35.772224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:35.772224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:35.772224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:35.772224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:35.772224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:35.772224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:35.772224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:35.772224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:35.772224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:35.772224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:35.772224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 242/483] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:35.778356 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415d5eb-9e4f-4a90-b334-a20e38de43e1 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:35.803658 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087afc80-9879-47bb-8453-7b452f42e62a None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:35.803658 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087afc80-9879-47bb-8453-7b452f42e62a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:35.803658 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087afc80-9879-47bb-8453-7b452f42e62a None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:35.805682 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 243/484] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 18:59:35.814669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2edb12-155d-47a7-9953-35817c879355 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:35.814953 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2edb12-155d-47a7-9953-35817c879355 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:35.815228 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2edb12-155d-47a7-9953-35817c879355 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:35.825780 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:35.843095 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4e2edb12-155d-47a7-9953-35817c879355 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:35.844546 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4e2edb12-155d-47a7-9953-35817c879355 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:35.901317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415d5eb-9e4f-4a90-b334-a20e38de43e1 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:35.910821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:35.951209 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070df16-17be-42ea-9dd7-c959a06df8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:35.951817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070df16-17be-42ea-9dd7-c959a06df8fe None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:35.952093 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070df16-17be-42ea-9dd7-c959a06df8fe None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:35.952361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070df16-17be-42ea-9dd7-c959a06df8fe None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:35.953018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070df16-17be-42ea-9dd7-c959a06df8fe None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:35.953341 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070df16-17be-42ea-9dd7-c959a06df8fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:35.954696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070df16-17be-42ea-9dd7-c959a06df8fe None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:35.978189 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 242/485] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 18:59:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:35.981941 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 244/486] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:35.982862 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070df16-17be-42ea-9dd7-c959a06df8fe None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:36.107479 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070df16-17be-42ea-9dd7-c959a06df8fe None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:36.143514 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583eab2-48c6-475e-b162-1ba02c50b587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:36.144145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583eab2-48c6-475e-b162-1ba02c50b587 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:36.144420 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583eab2-48c6-475e-b162-1ba02c50b587 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:36.144691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583eab2-48c6-475e-b162-1ba02c50b587 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:36.156333 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d583eab2-48c6-475e-b162-1ba02c50b587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:36.157751 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d583eab2-48c6-475e-b162-1ba02c50b587 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:36.172956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-d583eab2-48c6-475e-b162-1ba02c50b587 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:36.172956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:36.172956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:36.172956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:36.172956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:36.172956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:36.172956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:36.172956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:36.172956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:36.172956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:36.172956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:36.172956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:36.172956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:36.172956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:36.172956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:36.172956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:36.174865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:36.175697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:36.175697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:36.175697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:36.175697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 243/487] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:36.180003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2edb12-155d-47a7-9953-35817c879355 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:36.238990 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b90a3-d4f8-4a7d-86c0-ee42628460ea None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:36.239528 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b90a3-d4f8-4a7d-86c0-ee42628460ea None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:36.239945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b90a3-d4f8-4a7d-86c0-ee42628460ea None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:36.241462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 244/488] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:36.246141 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79432ea-818e-4e2f-af06-e15d27c5bf5c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:36.246584 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79432ea-818e-4e2f-af06-e15d27c5bf5c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:36.247012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79432ea-818e-4e2f-af06-e15d27c5bf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:36.280495 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f79432ea-818e-4e2f-af06-e15d27c5bf5c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:36.282912 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f79432ea-818e-4e2f-af06-e15d27c5bf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:36.318532 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:36.355489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2edb12-155d-47a7-9953-35817c879355 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:36.398360 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:36.406705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653db962-0918-4322-a3fc-31d87c1c0a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:36.407242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653db962-0918-4322-a3fc-31d87c1c0a7a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:36.407522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653db962-0918-4322-a3fc-31d87c1c0a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:36.407790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653db962-0918-4322-a3fc-31d87c1c0a7a None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:36.408429 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653db962-0918-4322-a3fc-31d87c1c0a7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:36.408820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653db962-0918-4322-a3fc-31d87c1c0a7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:36.410410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653db962-0918-4322-a3fc-31d87c1c0a7a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:36.410586 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 245/489] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:36.411635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79432ea-818e-4e2f-af06-e15d27c5bf5c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:36.429109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 245/490] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:36.430359 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653db962-0918-4322-a3fc-31d87c1c0a7a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:36.539849 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653db962-0918-4322-a3fc-31d87c1c0a7a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:36.581950 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429b1fd-4ec1-4d51-93b7-f38d73fef271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:36.582579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429b1fd-4ec1-4d51-93b7-f38d73fef271 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:36.582860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429b1fd-4ec1-4d51-93b7-f38d73fef271 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:36.583130 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429b1fd-4ec1-4d51-93b7-f38d73fef271 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:36.595035 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429b1fd-4ec1-4d51-93b7-f38d73fef271 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:36.596537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429b1fd-4ec1-4d51-93b7-f38d73fef271 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:36.609112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-a429b1fd-4ec1-4d51-93b7-f38d73fef271 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 18:59:36.609112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:36.609112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:36.609112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:36.609112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:36.609112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:36.609112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:36.609112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:36.609112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:36.609112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:36.609112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:36.609112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:36.609112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 18:59:36.609112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 18:59:36.609112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 18:59:36.609112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:36.610010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:36.611249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:36.611249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:36.611249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 18:59:36.611249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 18:59:36.611249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 18:59:36.611249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 18:59:36.611249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 18:59:36.611249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 18:59:36.611249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 18:59:36.611249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:36.611249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 246/491] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 18:59:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 18:59:36.611838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c926b-36df-4e6a-bc8e-ad35eabde2d3 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:36.611903 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c926b-36df-4e6a-bc8e-ad35eabde2d3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:36.612227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208c926b-36df-4e6a-bc8e-ad35eabde2d3 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:36.614758 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 247/492] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:36.618354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c926b-36df-4e6a-bc8e-ad35eabde2d3 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:36.626381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79432ea-818e-4e2f-af06-e15d27c5bf5c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:36.663830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b1a97-c016-4a2d-a02e-91c0e4de0af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:36.664383 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b1a97-c016-4a2d-a02e-91c0e4de0af7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:36.664658 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b1a97-c016-4a2d-a02e-91c0e4de0af7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:36.664927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b1a97-c016-4a2d-a02e-91c0e4de0af7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:36.676269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b1a97-c016-4a2d-a02e-91c0e4de0af7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:36.678510 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:36.682564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-ff3b1a97-c016-4a2d-a02e-91c0e4de0af7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:36.682564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:36.682564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:36.682564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:36.682564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:36.682564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:36.682564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:36.682564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:36.682564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:36.682564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:36.682564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:36.682564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:36.682564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:36.682564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:36.682564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:36.682564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:36.683750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:36.683750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:36.683750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:36.683750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:36.683750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:36.683750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:36.683750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:36.683750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:36.683750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:36.683750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:36.683750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:36.683750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:36.683750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:36.683750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:36.683750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:36.683750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:36.684719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:36.684719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:36.684719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:36.684719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:36.684719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:36.684719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:36.684719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:36.684719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:36.684719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:36.684719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:36.684719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:36.684719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:36.687662 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 246/493] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:36.689269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a0e90-10a0-4023-a102-ef01232864fc None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:36.689747 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a0e90-10a0-4023-a102-ef01232864fc None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:36.690241 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a0e90-10a0-4023-a102-ef01232864fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:36.710234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-5a9a0e90-10a0-4023-a102-ef01232864fc None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:36.711950 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5a9a0e90-10a0-4023-a102-ef01232864fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:36.715847 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:36.736873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208c926b-36df-4e6a-bc8e-ad35eabde2d3 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:36.773410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece3848-25ab-493c-86f2-190f9825ce56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:36.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece3848-25ab-493c-86f2-190f9825ce56 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:36.774353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece3848-25ab-493c-86f2-190f9825ce56 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:36.774633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ece3848-25ab-493c-86f2-190f9825ce56 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:36.775356 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece3848-25ab-493c-86f2-190f9825ce56 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:36.775662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece3848-25ab-493c-86f2-190f9825ce56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:36.777117 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ece3848-25ab-493c-86f2-190f9825ce56 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:36.801278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 247/494] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:36.801278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a0e90-10a0-4023-a102-ef01232864fc None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:36.824762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 248/495] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 18:59:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:36.825784 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece3848-25ab-493c-86f2-190f9825ce56 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:36.901246 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a0e90-10a0-4023-a102-ef01232864fc None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:36.929381 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ece3848-25ab-493c-86f2-190f9825ce56 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:36.938475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f3a25-1792-4956-8138-fe35835c6cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:36.939082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f3a25-1792-4956-8138-fe35835c6cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:36.939381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f3a25-1792-4956-8138-fe35835c6cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:36.939730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171f3a25-1792-4956-8138-fe35835c6cf2 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:36.940384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f3a25-1792-4956-8138-fe35835c6cf2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:36.940683 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f3a25-1792-4956-8138-fe35835c6cf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:36.941735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:36.943176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171f3a25-1792-4956-8138-fe35835c6cf2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:36.953343 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 248/496] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 18:59:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:36.954606 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f3a25-1792-4956-8138-fe35835c6cf2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:36.959536 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:36.980343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f414e-01c4-4306-9e5f-52284393c904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:36.981080 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f414e-01c4-4306-9e5f-52284393c904 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:36.981410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f414e-01c4-4306-9e5f-52284393c904 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:36.981851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f414e-01c4-4306-9e5f-52284393c904 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:36.995630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f414e-01c4-4306-9e5f-52284393c904 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:37.004222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-666f414e-01c4-4306-9e5f-52284393c904 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:37.004222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:37.004222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:37.004222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:37.004222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:37.004222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:37.004222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:37.004222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:37.004222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:37.004222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:37.004222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:37.004222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:37.004222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:37.004222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:37.004222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:37.004222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:37.005099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:37.005099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:37.005099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:37.005099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:37.005099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:37.005099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:37.005099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:37.005099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:37.005099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:37.005099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:37.005099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:37.005099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:37.005099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:37.005099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:37.005099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:37.005099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:37.006033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:37.006033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:37.006033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:37.006033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:37.006033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:37.006033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:37.006033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:37.006033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:37.006033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:37.006033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:37.006033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:37.006033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:37.006033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 249/497] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:37.006833 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f414e-01c4-4306-9e5f-52284393c904 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:37.080484 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171f3a25-1792-4956-8138-fe35835c6cf2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:37.117079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805da4a-276a-496c-a46e-2d2855271b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:37.117468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f414e-01c4-4306-9e5f-52284393c904 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:37.119172 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805da4a-276a-496c-a46e-2d2855271b6a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:37.119355 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805da4a-276a-496c-a46e-2d2855271b6a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:37.119623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0805da4a-276a-496c-a46e-2d2855271b6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:37.153119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5822e-aebe-4592-8bde-519616a5dc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:37.153771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5822e-aebe-4592-8bde-519616a5dc98 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:37.154048 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5822e-aebe-4592-8bde-519616a5dc98 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:37.154316 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5822e-aebe-4592-8bde-519616a5dc98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:37.165438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5822e-aebe-4592-8bde-519616a5dc98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:37.166887 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5822e-aebe-4592-8bde-519616a5dc98 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:37.181747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-56f5822e-aebe-4592-8bde-519616a5dc98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:37.181747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:37.181747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:37.181747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:37.181747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:37.181747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:37.181747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:37.181747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:37.181747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:37.181747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:37.181747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:37.181747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:37.181747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:37.181747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:37.181747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:37.181747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:37.183951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:37.185237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:37.185237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:37.185237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:37.185237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 250/498] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:37.186003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5822e-aebe-4592-8bde-519616a5dc98 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:37.196617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805da4a-276a-496c-a46e-2d2855271b6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:37.198123 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0805da4a-276a-496c-a46e-2d2855271b6a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:37.200297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:37.216280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:37.299174 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5822e-aebe-4592-8bde-519616a5dc98 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:37.309131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 249/499] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:37.312020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805da4a-276a-496c-a46e-2d2855271b6a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:37.333855 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f826c-dabb-44f8-a97b-d52829ef8aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:37.334443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f826c-dabb-44f8-a97b-d52829ef8aff None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:37.334716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f826c-dabb-44f8-a97b-d52829ef8aff None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:37.334986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15f826c-dabb-44f8-a97b-d52829ef8aff None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:37.335625 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f826c-dabb-44f8-a97b-d52829ef8aff None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:37.335909 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f826c-dabb-44f8-a97b-d52829ef8aff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:37.337282 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15f826c-dabb-44f8-a97b-d52829ef8aff None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:37.344780 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 251/500] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 18:59:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:37.345560 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f826c-dabb-44f8-a97b-d52829ef8aff None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:37.414984 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0805da4a-276a-496c-a46e-2d2855271b6a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:37.416572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:37.438144 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:37.452035 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe6d3d0-ac9f-444b-b62f-3f1ca3a52785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:37.452658 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6d3d0-ac9f-444b-b62f-3f1ca3a52785 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:37.452932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6d3d0-ac9f-444b-b62f-3f1ca3a52785 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:37.453218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe6d3d0-ac9f-444b-b62f-3f1ca3a52785 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:37.453396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15f826c-dabb-44f8-a97b-d52829ef8aff None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:37.463062 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe6d3d0-ac9f-444b-b62f-3f1ca3a52785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:37.468273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-2fe6d3d0-ac9f-444b-b62f-3f1ca3a52785 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:37.468273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:37.468273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:37.468273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:37.468273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:37.468273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:37.468273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:37.468273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:37.468273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:37.468273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:37.468273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:37.468273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:37.468273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:37.468273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:37.468273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:37.468273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:37.469176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:37.469176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:37.469176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:37.469176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:37.469176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:37.469176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:37.469176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:37.469176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:37.469176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:37.469176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:37.469176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:37.469176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:37.469176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:37.469176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:37.469176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:37.469176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:37.475294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:37.475294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:37.475294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:37.475294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:37.475294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:37.475294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:37.475294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:37.475294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:37.475294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:37.475294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:37.475294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:37.475294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:37.475294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 250/501] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:37.475294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796ca56-0542-41c9-8f70-1060b0f84b40 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:37.476515 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796ca56-0542-41c9-8f70-1060b0f84b40 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:37.476515 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c796ca56-0542-41c9-8f70-1060b0f84b40 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:37.476515 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 251/502] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:37.476515 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c796ca56-0542-41c9-8f70-1060b0f84b40 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:37.490350 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138e87f-c220-4dc7-be5b-cf98f4632bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:37.490952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138e87f-c220-4dc7-be5b-cf98f4632bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:37.491229 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138e87f-c220-4dc7-be5b-cf98f4632bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:37.491494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b138e87f-c220-4dc7-be5b-cf98f4632bd0 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:37.492119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138e87f-c220-4dc7-be5b-cf98f4632bd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:37.492417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138e87f-c220-4dc7-be5b-cf98f4632bd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:37.493791 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b138e87f-c220-4dc7-be5b-cf98f4632bd0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:37.505955 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 252/503] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:37.506584 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138e87f-c220-4dc7-be5b-cf98f4632bd0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:37.575363 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c796ca56-0542-41c9-8f70-1060b0f84b40 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:37.604898 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b138e87f-c220-4dc7-be5b-cf98f4632bd0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:37.608879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22346f32-305c-4ae3-8fcc-11e68d31468f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:37.609529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22346f32-305c-4ae3-8fcc-11e68d31468f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:37.609804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22346f32-305c-4ae3-8fcc-11e68d31468f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:37.610069 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22346f32-305c-4ae3-8fcc-11e68d31468f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:37.619975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22346f32-305c-4ae3-8fcc-11e68d31468f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:37.621541 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22346f32-305c-4ae3-8fcc-11e68d31468f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:37.636979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-22346f32-305c-4ae3-8fcc-11e68d31468f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:37.636979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:37.636979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:37.636979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:37.636979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:37.636979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:37.636979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:37.636979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:37.636979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:37.636979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:37.636979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:37.636979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:37.636979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:37.636979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:37.636979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:37.636979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:37.638068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:37.639106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:37.639106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:37.639106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:37.639106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 252/504] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:37.639337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22346f32-305c-4ae3-8fcc-11e68d31468f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:37.644107 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d75f5a-e001-41c6-b41e-a99ca4c74fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:37.644722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d75f5a-e001-41c6-b41e-a99ca4c74fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:37.645002 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d75f5a-e001-41c6-b41e-a99ca4c74fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:37.645297 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d75f5a-e001-41c6-b41e-a99ca4c74fd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:37.655271 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:37.656098 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d75f5a-e001-41c6-b41e-a99ca4c74fd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:37.657438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d75f5a-e001-41c6-b41e-a99ca4c74fd0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:37.660007 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:37.673804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-a2d75f5a-e001-41c6-b41e-a99ca4c74fd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:37.673804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:37.673804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:37.673804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:37.673804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:37.673804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:37.673804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:37.673804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:37.673804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:37.673804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:37.673804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:37.673804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:37.673804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:37.673804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:37.673804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:37.673804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:37.674886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:37.675837 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:37.675837 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:37.675837 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:37.675837 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 253/505] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:37.676308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab63eb-fe30-4234-a31c-3842487a7065 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:37.676582 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab63eb-fe30-4234-a31c-3842487a7065 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:37.676850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab63eb-fe30-4234-a31c-3842487a7065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:37.693957 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4eab63eb-fe30-4234-a31c-3842487a7065 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:37.695068 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4eab63eb-fe30-4234-a31c-3842487a7065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:37.746037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22346f32-305c-4ae3-8fcc-11e68d31468f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:37.760359 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 254/506] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:37.761198 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab63eb-fe30-4234-a31c-3842487a7065 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:37.785465 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc28be-674f-4682-ab2e-797b6de17f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:37.786357 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc28be-674f-4682-ab2e-797b6de17f11 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:37.786640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc28be-674f-4682-ab2e-797b6de17f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:37.786909 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc28be-674f-4682-ab2e-797b6de17f11 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:37.787588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc28be-674f-4682-ab2e-797b6de17f11 None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:37.787859 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc28be-674f-4682-ab2e-797b6de17f11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:37.789671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc28be-674f-4682-ab2e-797b6de17f11 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:37.805246 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 253/507] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 18:59:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:37.805246 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc28be-674f-4682-ab2e-797b6de17f11 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:37.869306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab63eb-fe30-4234-a31c-3842487a7065 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:37.895383 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:37.897328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:37.908913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a723eb-daa5-40e9-aa9a-fe0b92ea42c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:37.909714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a723eb-daa5-40e9-aa9a-fe0b92ea42c2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:37.909994 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a723eb-daa5-40e9-aa9a-fe0b92ea42c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:37.910263 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a723eb-daa5-40e9-aa9a-fe0b92ea42c2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:37.917423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a723eb-daa5-40e9-aa9a-fe0b92ea42c2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:37.918940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a723eb-daa5-40e9-aa9a-fe0b92ea42c2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:37.920834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc28be-674f-4682-ab2e-797b6de17f11 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:37.930319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-31a723eb-daa5-40e9-aa9a-fe0b92ea42c2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 18:59:37.930319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:37.930319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:37.930319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:37.930319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:37.930319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:37.930319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:37.930319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:37.930319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:37.930319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:37.930319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:37.930319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:37.930319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 18:59:37.930319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 18:59:37.930319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 18:59:37.930319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:37.931220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:37.932113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:37.932113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:37.932113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 18:59:37.932113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 18:59:37.932113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 18:59:37.932113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 18:59:37.932113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 18:59:37.932113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 18:59:37.932113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 18:59:37.932113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:37.932113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 255/508] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:37.932113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a723eb-daa5-40e9-aa9a-fe0b92ea42c2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:37.963404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b400f14c-0cb1-4622-8c86-178e7a1820e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:37.964040 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400f14c-0cb1-4622-8c86-178e7a1820e9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:37.964313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400f14c-0cb1-4622-8c86-178e7a1820e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:37.965123 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b400f14c-0cb1-4622-8c86-178e7a1820e9 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:37.965908 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b400f14c-0cb1-4622-8c86-178e7a1820e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:37.966254 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b400f14c-0cb1-4622-8c86-178e7a1820e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:37.967674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b400f14c-0cb1-4622-8c86-178e7a1820e9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:37.985580 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 254/509] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:37.985580 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b400f14c-0cb1-4622-8c86-178e7a1820e9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:38.041877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a723eb-daa5-40e9-aa9a-fe0b92ea42c2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:38.077123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de126c76-dd0b-4bd0-a0e9-a3b8b8485327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:38.077762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de126c76-dd0b-4bd0-a0e9-a3b8b8485327 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:38.078041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de126c76-dd0b-4bd0-a0e9-a3b8b8485327 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:38.078312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de126c76-dd0b-4bd0-a0e9-a3b8b8485327 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:38.078938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de126c76-dd0b-4bd0-a0e9-a3b8b8485327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:38.079236 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de126c76-dd0b-4bd0-a0e9-a3b8b8485327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:38.080585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de126c76-dd0b-4bd0-a0e9-a3b8b8485327 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:38.082642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b400f14c-0cb1-4622-8c86-178e7a1820e9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:38.092713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 256/510] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:38.093519 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de126c76-dd0b-4bd0-a0e9-a3b8b8485327 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:38.119771 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91e4c1-e637-49d7-a8ac-c1ca69257c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:38.120404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91e4c1-e637-49d7-a8ac-c1ca69257c40 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:38.120680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91e4c1-e637-49d7-a8ac-c1ca69257c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:38.120976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c91e4c1-e637-49d7-a8ac-c1ca69257c40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:38.123920 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:38.131798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91e4c1-e637-49d7-a8ac-c1ca69257c40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:38.133245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c91e4c1-e637-49d7-a8ac-c1ca69257c40 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:38.139155 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:38.148491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-9c91e4c1-e637-49d7-a8ac-c1ca69257c40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:38.148491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:38.148491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:38.148491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:38.148491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:38.148491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:38.148491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:38.148491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:38.148491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:38.148491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:38.148491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:38.148491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:38.148491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:38.148491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:38.148491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:38.148491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:38.149602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:38.150472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:38.150472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:38.150472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:38.150472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 255/511] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:38.150472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91e4c1-e637-49d7-a8ac-c1ca69257c40 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:38.195997 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de126c76-dd0b-4bd0-a0e9-a3b8b8485327 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:38.230410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f1ba2-03c0-48e5-acc4-7dea1fa1abcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:38.230934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f1ba2-03c0-48e5-acc4-7dea1fa1abcd None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:38.231212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f1ba2-03c0-48e5-acc4-7dea1fa1abcd None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:38.231479 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f1ba2-03c0-48e5-acc4-7dea1fa1abcd None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:38.232194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f1ba2-03c0-48e5-acc4-7dea1fa1abcd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:38.232491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f1ba2-03c0-48e5-acc4-7dea1fa1abcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:38.233916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f1ba2-03c0-48e5-acc4-7dea1fa1abcd None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:38.251033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 257/512] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 31 18:59:38 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:38.251579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c91e4c1-e637-49d7-a8ac-c1ca69257c40 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:38.252119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f1ba2-03c0-48e5-acc4-7dea1fa1abcd None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:38.285040 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e0a01-b064-4856-9ba1-1f8d10a341a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:38.285688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e0a01-b064-4856-9ba1-1f8d10a341a7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:38.285963 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e0a01-b064-4856-9ba1-1f8d10a341a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:38.286230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887e0a01-b064-4856-9ba1-1f8d10a341a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:38.292698 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e0a01-b064-4856-9ba1-1f8d10a341a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:38.294151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887e0a01-b064-4856-9ba1-1f8d10a341a7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:38.305727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-887e0a01-b064-4856-9ba1-1f8d10a341a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:38.305727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:38.305727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:38.305727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:38.305727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:38.305727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:38.305727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:38.305727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:38.305727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:38.305727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:38.305727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:38.305727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:38.305727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 18:59:38.305727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 18:59:38.305727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:38.306574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:38.307453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 18:59:38.307453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 18:59:38.307453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 18:59:38.307453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 18:59:38.307453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 18:59:38.307453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 18:59:38.307453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:38.307453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:38.307453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 256/513] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:38.307927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e0a01-b064-4856-9ba1-1f8d10a341a7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:38.352675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f1ba2-03c0-48e5-acc4-7dea1fa1abcd None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:38.355185 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:38.356029 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:38.388393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38448075-f0de-4b7b-b681-e6f867bd476f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:38.389012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38448075-f0de-4b7b-b681-e6f867bd476f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:38.389411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38448075-f0de-4b7b-b681-e6f867bd476f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:38.389597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38448075-f0de-4b7b-b681-e6f867bd476f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:38.400430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38448075-f0de-4b7b-b681-e6f867bd476f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:38.401905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38448075-f0de-4b7b-b681-e6f867bd476f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:38.407240 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887e0a01-b064-4856-9ba1-1f8d10a341a7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:38.416131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-38448075-f0de-4b7b-b681-e6f867bd476f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:38.416131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:38.416131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:38.416131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:38.416131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:38.416131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:38.416131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:38.416131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:38.416131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:38.416131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:38.416131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:38.416131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:38.416131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:38.416131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:38.416131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:38.416131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:38.417216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:38.418262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:38.418262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:38.418262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:38.418262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 258/514] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:38.418262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38448075-f0de-4b7b-b681-e6f867bd476f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:38.440774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3956c-4401-49b3-931e-a99d80954bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:38.441420 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3956c-4401-49b3-931e-a99d80954bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:38.441696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3956c-4401-49b3-931e-a99d80954bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:38.441962 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb3956c-4401-49b3-931e-a99d80954bb5 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:38.449833 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3956c-4401-49b3-931e-a99d80954bb5 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:38.451260 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb3956c-4401-49b3-931e-a99d80954bb5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:38.463297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-dfb3956c-4401-49b3-931e-a99d80954bb5 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 18:59:38.463297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:38.463297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:38.463297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:38.463297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:38.463297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:38.463297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:38.463297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:38.463297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:38.463297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:38.463297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:38.463297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:38.463297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 18:59:38.463297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 18:59:38.463297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 18:59:38.463297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:38.464366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:38.465434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:38.465434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:38.465434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 18:59:38.465434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 18:59:38.465434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 18:59:38.465434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 18:59:38.465434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 18:59:38.465434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 18:59:38.465434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 18:59:38.465434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:38.465434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 257/515] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:38.465434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3956c-4401-49b3-931e-a99d80954bb5 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:38.524838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38448075-f0de-4b7b-b681-e6f867bd476f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:38.557024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109c5f9-0045-4b92-a022-3da377de57fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:38.557633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109c5f9-0045-4b92-a022-3da377de57fd None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:38.557905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109c5f9-0045-4b92-a022-3da377de57fd None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:38.558169 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c109c5f9-0045-4b92-a022-3da377de57fd None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:38.559317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109c5f9-0045-4b92-a022-3da377de57fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:38.559614 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109c5f9-0045-4b92-a022-3da377de57fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:38.560948 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c109c5f9-0045-4b92-a022-3da377de57fd None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:38.571160 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb3956c-4401-49b3-931e-a99d80954bb5 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:38.572097 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 259/516] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:38.572863 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109c5f9-0045-4b92-a022-3da377de57fd None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:38.580814 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:38.590645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:38.606876 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b5ef4-2129-4b55-9bb5-6f122f734999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:38.607472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b5ef4-2129-4b55-9bb5-6f122f734999 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:38.607747 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b5ef4-2129-4b55-9bb5-6f122f734999 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:38.608016 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b5ef4-2129-4b55-9bb5-6f122f734999 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:38.673089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c109c5f9-0045-4b92-a022-3da377de57fd None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:38.673900 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b5ef4-2129-4b55-9bb5-6f122f734999 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:38.675435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b5ef4-2129-4b55-9bb5-6f122f734999 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:38.707042 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1874a6b-d143-417a-be81-9c2169addd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:38.707635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1874a6b-d143-417a-be81-9c2169addd13 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:38.707912 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1874a6b-d143-417a-be81-9c2169addd13 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:38.708181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1874a6b-d143-417a-be81-9c2169addd13 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:38.708856 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1874a6b-d143-417a-be81-9c2169addd13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:38.709157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1874a6b-d143-417a-be81-9c2169addd13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:38.710586 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1874a6b-d143-417a-be81-9c2169addd13 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:38.727770 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 260/517] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 18:59:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:38.728552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1874a6b-d143-417a-be81-9c2169addd13 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:38.775754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 258/518] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:38.776794 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b5ef4-2129-4b55-9bb5-6f122f734999 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:38.789662 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:38.805069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:38.828284 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1874a6b-d143-417a-be81-9c2169addd13 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:38.860604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5762bf-5ab6-4071-a1a9-481793dd24ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:38.861211 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5762bf-5ab6-4071-a1a9-481793dd24ef None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:38.861495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5762bf-5ab6-4071-a1a9-481793dd24ef None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:38.861763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5762bf-5ab6-4071-a1a9-481793dd24ef None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:38.862385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5762bf-5ab6-4071-a1a9-481793dd24ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:38.862713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5762bf-5ab6-4071-a1a9-481793dd24ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:38.864175 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5762bf-5ab6-4071-a1a9-481793dd24ef None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:38.876332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 261/519] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:38.877089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5762bf-5ab6-4071-a1a9-481793dd24ef None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:38.877825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b5ef4-2129-4b55-9bb5-6f122f734999 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:38.913809 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf19989-66ca-495b-97cf-24f7d1cc0a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:38.914402 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf19989-66ca-495b-97cf-24f7d1cc0a49 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:38.914727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf19989-66ca-495b-97cf-24f7d1cc0a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:38.914993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf19989-66ca-495b-97cf-24f7d1cc0a49 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:38.915728 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf19989-66ca-495b-97cf-24f7d1cc0a49 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:38.916030 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf19989-66ca-495b-97cf-24f7d1cc0a49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:38.917460 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf19989-66ca-495b-97cf-24f7d1cc0a49 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:38.954150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 259/520] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 31 18:59:38 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:38.955116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf19989-66ca-495b-97cf-24f7d1cc0a49 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:38.974775 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5762bf-5ab6-4071-a1a9-481793dd24ef None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:39.008838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f78a8-74af-4d69-9f44-f4399ad61016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:39.009489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f78a8-74af-4d69-9f44-f4399ad61016 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:39.009769 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f78a8-74af-4d69-9f44-f4399ad61016 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:39.010039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256f78a8-74af-4d69-9f44-f4399ad61016 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:39.012645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:39.018022 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256f78a8-74af-4d69-9f44-f4399ad61016 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:39.019526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256f78a8-74af-4d69-9f44-f4399ad61016 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:39.020283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:39.032496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-256f78a8-74af-4d69-9f44-f4399ad61016 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 18:59:39.032496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:39.032496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:39.032496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:39.032496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:39.032496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:39.032496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:39.032496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:39.032496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:39.032496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:39.032496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:39.032496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:39.032496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 18:59:39.032496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 18:59:39.032496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 18:59:39.032496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:39.038266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:39.039431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:39.039431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:39.039431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 18:59:39.039431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 18:59:39.039431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 18:59:39.039431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 18:59:39.039431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 18:59:39.039431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 18:59:39.039431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 18:59:39.039431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:39.039431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 262/521] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:39.039431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f78a8-74af-4d69-9f44-f4399ad61016 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:39.055158 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf19989-66ca-495b-97cf-24f7d1cc0a49 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:39.087731 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5e94c-b5f6-45f1-9642-0309ec2fdf44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:39.088328 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5e94c-b5f6-45f1-9642-0309ec2fdf44 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:39.088613 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5e94c-b5f6-45f1-9642-0309ec2fdf44 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:39.088886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c5e94c-b5f6-45f1-9642-0309ec2fdf44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:39.090166 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5e94c-b5f6-45f1-9642-0309ec2fdf44 None admin] RBAC: Adding query filter params (role.id=6e93fc836fe74c9daa11c7b2848a2922, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=673a72ac9a9847e3812007b66ac7a5ea) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:39.090474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5e94c-b5f6-45f1-9642-0309ec2fdf44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:39.091890 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c5e94c-b5f6-45f1-9642-0309ec2fdf44 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:39.097446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 260/522] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:59:38 2022] GET /identity/v3/role_assignments?user.id=673a72ac9a9847e3812007b66ac7a5ea&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=6e93fc836fe74c9daa11c7b2848a2922 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:39.098191 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5e94c-b5f6-45f1-9642-0309ec2fdf44 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:39.139698 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256f78a8-74af-4d69-9f44-f4399ad61016 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:39.178302 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9060cf3-fca0-46ed-bd51-054b60da24f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:39.178934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9060cf3-fca0-46ed-bd51-054b60da24f0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:39.179207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9060cf3-fca0-46ed-bd51-054b60da24f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:39.179473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9060cf3-fca0-46ed-bd51-054b60da24f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:39.192933 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9060cf3-fca0-46ed-bd51-054b60da24f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:39.192933 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9060cf3-fca0-46ed-bd51-054b60da24f0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:39.208566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-a9060cf3-fca0-46ed-bd51-054b60da24f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:39.208566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:39.208566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:39.208566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:39.208566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:39.208566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:39.208566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:39.208566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:39.208566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:39.208566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:39.208566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:39.208566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:39.208566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:39.208566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:39.208566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:39.208566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:39.210187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:39.211179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:39.211179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:39.211179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:39.215671 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 263/523] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:39.216552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9060cf3-fca0-46ed-bd51-054b60da24f0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:39.229369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c5e94c-b5f6-45f1-9642-0309ec2fdf44 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:39.262255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:39.269599 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fbaaf-11df-4071-8599-3aa0cfffa80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:39.270208 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fbaaf-11df-4071-8599-3aa0cfffa80e None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:39.270488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fbaaf-11df-4071-8599-3aa0cfffa80e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:39.270762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617fbaaf-11df-4071-8599-3aa0cfffa80e None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:39.271406 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fbaaf-11df-4071-8599-3aa0cfffa80e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:39.271744 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:39.272848 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617fbaaf-11df-4071-8599-3aa0cfffa80e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:39.305309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 261/524] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 18:59:39 2022] POST /identity/v3/users => generated 247 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:59:39.305309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fbaaf-11df-4071-8599-3aa0cfffa80e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:39.347377 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9060cf3-fca0-46ed-bd51-054b60da24f0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:39.399622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797036ab-a992-4e4d-812c-e513c9298860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:39.400340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797036ab-a992-4e4d-812c-e513c9298860 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:39.401271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797036ab-a992-4e4d-812c-e513c9298860 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:39.401622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797036ab-a992-4e4d-812c-e513c9298860 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:39.435288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617fbaaf-11df-4071-8599-3aa0cfffa80e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:39.481939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672c909-768c-4394-b36a-328b3d27200b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:39.482556 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672c909-768c-4394-b36a-328b3d27200b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:39.482831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672c909-768c-4394-b36a-328b3d27200b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:39.483116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c672c909-768c-4394-b36a-328b3d27200b None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:39.483864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672c909-768c-4394-b36a-328b3d27200b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:39.484166 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672c909-768c-4394-b36a-328b3d27200b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:39.485888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c672c909-768c-4394-b36a-328b3d27200b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:39.497985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797036ab-a992-4e4d-812c-e513c9298860 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:39.500547 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797036ab-a992-4e4d-812c-e513c9298860 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:39.507379 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 262/525] 10.222.0.65 () {56 vars in 1139 bytes} [Wed Aug 31 18:59:39 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:39.510583 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672c909-768c-4394-b36a-328b3d27200b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:39.559738 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:39.564836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:39.622114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 264/526] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:39.622983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797036ab-a992-4e4d-812c-e513c9298860 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:39.634195 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c672c909-768c-4394-b36a-328b3d27200b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:39.681517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29540f33-f32d-4895-af82-1deafd99b441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:39.682346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29540f33-f32d-4895-af82-1deafd99b441 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:39.682805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29540f33-f32d-4895-af82-1deafd99b441 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:39.683192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29540f33-f32d-4895-af82-1deafd99b441 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:39.684113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29540f33-f32d-4895-af82-1deafd99b441 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:39.684485 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29540f33-f32d-4895-af82-1deafd99b441 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:39.686534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29540f33-f32d-4895-af82-1deafd99b441 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:39.705343 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 263/527] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:39.706233 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29540f33-f32d-4895-af82-1deafd99b441 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:39.734304 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797036ab-a992-4e4d-812c-e513c9298860 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:39.772522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095c828-34e2-437b-8c08-45090c4c8680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:39.773340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095c828-34e2-437b-8c08-45090c4c8680 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:39.773620 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095c828-34e2-437b-8c08-45090c4c8680 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:39.773885 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b095c828-34e2-437b-8c08-45090c4c8680 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:39.792787 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:39.850014 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b095c828-34e2-437b-8c08-45090c4c8680 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:39.851727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b095c828-34e2-437b-8c08-45090c4c8680 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:39.864121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29540f33-f32d-4895-af82-1deafd99b441 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:39.867130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:39.912201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a326f4-fea8-475e-99b4-d589da77f3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:39.913663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a326f4-fea8-475e-99b4-d589da77f3bb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:39.914362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a326f4-fea8-475e-99b4-d589da77f3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:39.914778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a326f4-fea8-475e-99b4-d589da77f3bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:39.929613 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a326f4-fea8-475e-99b4-d589da77f3bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:39.932393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a326f4-fea8-475e-99b4-d589da77f3bb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:39.951400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-a9a326f4-fea8-475e-99b4-d589da77f3bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:39.951400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:39.951400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:39.951400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:39.951400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:39.951400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:39.951400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:39.951400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:39.951400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:39.951400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:39.951400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:39.951400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:39.951400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:39.951400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:39.951400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:39.951400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:39.952546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:39.953634 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:39.953634 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:39.953634 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:39.955796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 264/528] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:39.957253 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a326f4-fea8-475e-99b4-d589da77f3bb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:39.971697 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 265/529] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:39.972759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095c828-34e2-437b-8c08-45090c4c8680 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:40.026968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:40.084247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b095c828-34e2-437b-8c08-45090c4c8680 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:40.096642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a326f4-fea8-475e-99b4-d589da77f3bb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:40.122029 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfc5f1-a89a-43ee-afce-85d18aa0497e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:40.122638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfc5f1-a89a-43ee-afce-85d18aa0497e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:40.123011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfc5f1-a89a-43ee-afce-85d18aa0497e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:40.123349 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfc5f1-a89a-43ee-afce-85d18aa0497e None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:40.124328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfc5f1-a89a-43ee-afce-85d18aa0497e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:40.124692 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfc5f1-a89a-43ee-afce-85d18aa0497e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:40.126314 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfc5f1-a89a-43ee-afce-85d18aa0497e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:40.139141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 266/530] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:40.140208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfc5f1-a89a-43ee-afce-85d18aa0497e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:40.158561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f8611-5eb9-4f2b-b68a-af5e139b7e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:40.159870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f8611-5eb9-4f2b-b68a-af5e139b7e6e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:40.160326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f8611-5eb9-4f2b-b68a-af5e139b7e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:40.160802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f8611-5eb9-4f2b-b68a-af5e139b7e6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:40.171003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f8611-5eb9-4f2b-b68a-af5e139b7e6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:40.173654 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f8611-5eb9-4f2b-b68a-af5e139b7e6e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:40.190201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-383f8611-5eb9-4f2b-b68a-af5e139b7e6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:40.190201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:40.190201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:40.190201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:40.190201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:40.190201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:40.190201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:40.190201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:40.190201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:40.190201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:40.190201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:40.190201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:40.190201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 18:59:40.190201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 18:59:40.190201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:40.191222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:40.192088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 18:59:40.192088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 18:59:40.192088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 18:59:40.192088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 18:59:40.192088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 18:59:40.192088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 18:59:40.192088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:40.192088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:40.194384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 265/531] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:40.196960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f8611-5eb9-4f2b-b68a-af5e139b7e6e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:40.221821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:40.272520 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfc5f1-a89a-43ee-afce-85d18aa0497e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:40.284845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:40.314272 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f8611-5eb9-4f2b-b68a-af5e139b7e6e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:40.318987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209b1cf-898a-4450-9d0b-fccef2f4cb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:40.319707 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209b1cf-898a-4450-9d0b-fccef2f4cb00 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:40.320057 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209b1cf-898a-4450-9d0b-fccef2f4cb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:40.320401 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d209b1cf-898a-4450-9d0b-fccef2f4cb00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:40.332251 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209b1cf-898a-4450-9d0b-fccef2f4cb00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:40.333881 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d209b1cf-898a-4450-9d0b-fccef2f4cb00 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:40.349345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-d209b1cf-898a-4450-9d0b-fccef2f4cb00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:40.349345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:40.349345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:40.349345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:40.349345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:40.349345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:40.349345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:40.349345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:40.349345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:40.349345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:40.349345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:40.349345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:40.349345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:40.349345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:40.349345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:40.349345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:40.350221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:40.351291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:40.351291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:40.351291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:40.351291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 267/532] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:40.352211 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209b1cf-898a-4450-9d0b-fccef2f4cb00 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:40.358217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcb62f-30a7-481a-bcb3-95d1efd00d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:40.359058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcb62f-30a7-481a-bcb3-95d1efd00d0a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:40.359476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcb62f-30a7-481a-bcb3-95d1efd00d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:40.359885 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bcb62f-30a7-481a-bcb3-95d1efd00d0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:40.369115 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcb62f-30a7-481a-bcb3-95d1efd00d0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:40.371841 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bcb62f-30a7-481a-bcb3-95d1efd00d0a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:40.388645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-40bcb62f-30a7-481a-bcb3-95d1efd00d0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:40.388645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:40.388645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:40.388645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:40.388645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:40.388645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:40.388645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:40.388645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:40.388645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:40.388645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:40.388645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:40.388645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:40.388645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 18:59:40.388645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 18:59:40.388645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:40.389663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:40.390569 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 18:59:40.390569 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 18:59:40.390569 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 18:59:40.390569 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 18:59:40.390569 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 18:59:40.390569 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 18:59:40.390569 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:40.390569 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:40.393646 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 266/533] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:40.394670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcb62f-30a7-481a-bcb3-95d1efd00d0a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:40.467978 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d209b1cf-898a-4450-9d0b-fccef2f4cb00 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:40.498520 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:40.518960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e99fd9-38cf-42c7-9460-6c964f6b2ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:40.519571 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e99fd9-38cf-42c7-9460-6c964f6b2ffb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:40.519848 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e99fd9-38cf-42c7-9460-6c964f6b2ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:40.520116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e99fd9-38cf-42c7-9460-6c964f6b2ffb None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:40.520863 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e99fd9-38cf-42c7-9460-6c964f6b2ffb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:40.521166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e99fd9-38cf-42c7-9460-6c964f6b2ffb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:40.522711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e99fd9-38cf-42c7-9460-6c964f6b2ffb None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:40.531647 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bcb62f-30a7-481a-bcb3-95d1efd00d0a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:40.536819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 268/534] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 18:59:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:40.537768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e99fd9-38cf-42c7-9460-6c964f6b2ffb None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:40.565125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:40.572067 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4834d2-5761-46ec-9e58-5493a91cc78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:40.572783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4834d2-5761-46ec-9e58-5493a91cc78e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:40.573126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4834d2-5761-46ec-9e58-5493a91cc78e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:40.573441 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4834d2-5761-46ec-9e58-5493a91cc78e None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:40.574103 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4834d2-5761-46ec-9e58-5493a91cc78e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:40.574414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4834d2-5761-46ec-9e58-5493a91cc78e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:40.575886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4834d2-5761-46ec-9e58-5493a91cc78e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:40.590473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 267/535] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:40.591577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4834d2-5761-46ec-9e58-5493a91cc78e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:40.662761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e99fd9-38cf-42c7-9460-6c964f6b2ffb None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:40.706354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4834d2-5761-46ec-9e58-5493a91cc78e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:40.710373 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931ff8b-8656-4908-a24e-f0a95b7b0728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:40.710976 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931ff8b-8656-4908-a24e-f0a95b7b0728 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:40.711250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931ff8b-8656-4908-a24e-f0a95b7b0728 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:40.711517 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6931ff8b-8656-4908-a24e-f0a95b7b0728 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:40.712201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931ff8b-8656-4908-a24e-f0a95b7b0728 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:40.712499 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931ff8b-8656-4908-a24e-f0a95b7b0728 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:40.713966 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6931ff8b-8656-4908-a24e-f0a95b7b0728 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:40.732433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 269/536] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 18:59:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:40.733441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931ff8b-8656-4908-a24e-f0a95b7b0728 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:40.749650 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4992cfe-959a-4fd4-8aea-fcf0d4ead4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:40.750868 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4992cfe-959a-4fd4-8aea-fcf0d4ead4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:40.751529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4992cfe-959a-4fd4-8aea-fcf0d4ead4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:40.751942 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4992cfe-959a-4fd4-8aea-fcf0d4ead4f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:40.752963 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4992cfe-959a-4fd4-8aea-fcf0d4ead4f0 None admin] RBAC: Adding query filter params (role.id=6e93fc836fe74c9daa11c7b2848a2922, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=1863336fecad4987acdfd0b3f7b2f70f) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:40.753524 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4992cfe-959a-4fd4-8aea-fcf0d4ead4f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:40.755367 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4992cfe-959a-4fd4-8aea-fcf0d4ead4f0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:40.759218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:40.763936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 268/537] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:59:40 2022] GET /identity/v3/role_assignments?user.id=1863336fecad4987acdfd0b3f7b2f70f&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=6e93fc836fe74c9daa11c7b2848a2922 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:40.765984 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4992cfe-959a-4fd4-8aea-fcf0d4ead4f0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:40.825167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:40.848183 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6931ff8b-8656-4908-a24e-f0a95b7b0728 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:40.887046 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2da88-41a9-4879-88c3-193d087fa68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:40.887881 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2da88-41a9-4879-88c3-193d087fa68b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:40.888536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2da88-41a9-4879-88c3-193d087fa68b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:40.888536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb2da88-41a9-4879-88c3-193d087fa68b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:40.896471 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2da88-41a9-4879-88c3-193d087fa68b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:40.898200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb2da88-41a9-4879-88c3-193d087fa68b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:40.904495 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4992cfe-959a-4fd4-8aea-fcf0d4ead4f0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:40.916438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-fdb2da88-41a9-4879-88c3-193d087fa68b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:40.916438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:40.916438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:40.916438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:40.916438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:40.916438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:40.916438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:40.916438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:40.916438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:40.916438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:40.916438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:40.916438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:40.916438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 18:59:40.916438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 18:59:40.916438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:40.917513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:40.918396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 18:59:40.918396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 18:59:40.918396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 18:59:40.918396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 18:59:40.918396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 18:59:40.918396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 18:59:40.918396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:40.918396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:40.920783 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 270/538] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:40.928703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2da88-41a9-4879-88c3-193d087fa68b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:40.962218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dfa12-15f1-4850-bdb0-51ac77b0e759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:40.963603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dfa12-15f1-4850-bdb0-51ac77b0e759 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:40.964190 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dfa12-15f1-4850-bdb0-51ac77b0e759 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:40.965318 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dfa12-15f1-4850-bdb0-51ac77b0e759 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:40.966216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dfa12-15f1-4850-bdb0-51ac77b0e759 None admin] RBAC: Adding query filter params (role.id=6e93fc836fe74c9daa11c7b2848a2922, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=5d157abcf94745a5b92d6df64d872f73) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:40.966693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dfa12-15f1-4850-bdb0-51ac77b0e759 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:40.968511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dfa12-15f1-4850-bdb0-51ac77b0e759 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:40.976006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 269/539] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:59:40 2022] GET /identity/v3/role_assignments?user.id=5d157abcf94745a5b92d6df64d872f73&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=6e93fc836fe74c9daa11c7b2848a2922 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:41.078466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb2da88-41a9-4879-88c3-193d087fa68b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:41.120792 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c82eb60-5bb7-422c-ae00-8eca846d2941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:41.121436 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82eb60-5bb7-422c-ae00-8eca846d2941 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:41.121716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82eb60-5bb7-422c-ae00-8eca846d2941 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:41.121985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c82eb60-5bb7-422c-ae00-8eca846d2941 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:41.122681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82eb60-5bb7-422c-ae00-8eca846d2941 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:41.123048 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82eb60-5bb7-422c-ae00-8eca846d2941 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:41.124716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c82eb60-5bb7-422c-ae00-8eca846d2941 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:41.134504 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:41.145841 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 271/540] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 18:59:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:41.154988 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dfa12-15f1-4850-bdb0-51ac77b0e759 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:41.192397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0515a01-2e59-4b02-90e7-7162dc323488 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:41.192602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0515a01-2e59-4b02-90e7-7162dc323488 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:41.192885 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0515a01-2e59-4b02-90e7-7162dc323488 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:41.194466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 272/541] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:41.200492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bd59d-9332-4db3-ba46-14be15053fb6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:41.200801 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bd59d-9332-4db3-ba46-14be15053fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:41.201033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bd59d-9332-4db3-ba46-14be15053fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:41.225976 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e35bd59d-9332-4db3-ba46-14be15053fb6 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:41.227317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e35bd59d-9332-4db3-ba46-14be15053fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:41.261961 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:41.304910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dfa12-15f1-4850-bdb0-51ac77b0e759 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:41.336224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 273/542] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:59:41.359421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53688ec-25de-48f4-aaa2-26a1a002274e None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:41.359717 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53688ec-25de-48f4-aaa2-26a1a002274e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:41.359928 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53688ec-25de-48f4-aaa2-26a1a002274e None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:41.362421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 274/543] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 18:59:41.364631 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6801c-e982-4c21-ac2a-54d4d2ac29ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:41.369601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec17db-387c-480b-adcb-33954e7e61d3 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:41.369601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec17db-387c-480b-adcb-33954e7e61d3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:41.369601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ec17db-387c-480b-adcb-33954e7e61d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:41.371263 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6801c-e982-4c21-ac2a-54d4d2ac29ce None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:41.371746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6801c-e982-4c21-ac2a-54d4d2ac29ce None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:41.372327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd6801c-e982-4c21-ac2a-54d4d2ac29ce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:41.373288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6801c-e982-4c21-ac2a-54d4d2ac29ce None admin] RBAC: Adding query filter params (role.id=6e93fc836fe74c9daa11c7b2848a2922, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=093edc8c192c469f9532a45bb3326612) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:41.374175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6801c-e982-4c21-ac2a-54d4d2ac29ce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:41.375931 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd6801c-e982-4c21-ac2a-54d4d2ac29ce None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:41.387589 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f1ec17db-387c-480b-adcb-33954e7e61d3 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:41.388217 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f1ec17db-387c-480b-adcb-33954e7e61d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:41.389311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 270/544] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:59:41 2022] GET /identity/v3/role_assignments?user.id=093edc8c192c469f9532a45bb3326612&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=6e93fc836fe74c9daa11c7b2848a2922 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:41.460742 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 275/545] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:59:41.675418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6801c-e982-4c21-ac2a-54d4d2ac29ce None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:41.778438 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd6801c-e982-4c21-ac2a-54d4d2ac29ce None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:41.805600 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:41.814921 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f16201-128a-4b4b-9f3f-bd2af3ffecd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:41.815799 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f16201-128a-4b4b-9f3f-bd2af3ffecd2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:41.816245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f16201-128a-4b4b-9f3f-bd2af3ffecd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:41.816681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f16201-128a-4b4b-9f3f-bd2af3ffecd2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:41.818917 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec17db-387c-480b-adcb-33954e7e61d3 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:41.827974 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f16201-128a-4b4b-9f3f-bd2af3ffecd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:41.833422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-e4f16201-128a-4b4b-9f3f-bd2af3ffecd2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:41.833422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:41.833422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:41.833422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:41.833422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:41.833422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:41.833422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:41.833422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:41.833422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:41.833422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:41.833422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:41.833422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:41.833422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:41.833422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:41.833422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:41.833422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:41.834283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:41.834283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:41.834283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:41.834283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:41.834283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:41.834283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:41.834283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:41.834283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:41.834283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:41.834283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:41.834283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:41.834283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:41.834283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:41.834283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:41.834283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:41.834283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:41.835053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:41.835053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:41.835053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:41.835053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:41.835053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:41.835053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:41.835053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:41.835053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:41.835053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:41.835053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:41.835053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:41.835053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:41.837110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 271/546] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:41.842938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f16201-128a-4b4b-9f3f-bd2af3ffecd2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:41.943663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:41.961780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f16201-128a-4b4b-9f3f-bd2af3ffecd2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:42.017597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ec17db-387c-480b-adcb-33954e7e61d3 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:42.025963 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb6b09-9c83-43e7-b1a6-caceee54f385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:42.026580 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb6b09-9c83-43e7-b1a6-caceee54f385 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:42.026853 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb6b09-9c83-43e7-b1a6-caceee54f385 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:42.027647 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfb6b09-9c83-43e7-b1a6-caceee54f385 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:42.028312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb6b09-9c83-43e7-b1a6-caceee54f385 None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:42.028661 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb6b09-9c83-43e7-b1a6-caceee54f385 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:42.030012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfb6b09-9c83-43e7-b1a6-caceee54f385 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:42.044573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 272/547] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 18:59:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:42.051657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb6b09-9c83-43e7-b1a6-caceee54f385 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:42.079971 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108d74e-b587-49b1-a41d-5462dbb993ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:42.080604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108d74e-b587-49b1-a41d-5462dbb993ae None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:42.080881 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108d74e-b587-49b1-a41d-5462dbb993ae None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:42.081145 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108d74e-b587-49b1-a41d-5462dbb993ae None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:42.095558 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7108d74e-b587-49b1-a41d-5462dbb993ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:42.100491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-7108d74e-b587-49b1-a41d-5462dbb993ae None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:42.100491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:42.100491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:42.100491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:42.100491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:42.100491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:42.100491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:42.100491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:42.100491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:42.100491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:42.100491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:42.100491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:42.100491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:42.100491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:42.100491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:42.100491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:42.101414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:42.101414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:42.101414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:42.101414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:42.101414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:42.101414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:42.101414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:42.101414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:42.101414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:42.101414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:42.101414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:42.101414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:42.101414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:42.101414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:42.101414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:42.101414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:42.102193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:42.102193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:42.102193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:42.102193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:42.102193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:42.102193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:42.102193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:42.102193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:42.102193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:42.102193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:42.102193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:42.102193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:42.107290 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 276/548] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:42.108853 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108d74e-b587-49b1-a41d-5462dbb993ae None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:42.142018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:42.232642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108d74e-b587-49b1-a41d-5462dbb993ae None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:42.244000 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfb6b09-9c83-43e7-b1a6-caceee54f385 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:42.277887 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:42.300049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30e119-4345-40a2-ad24-9c96024c9c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:42.300655 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30e119-4345-40a2-ad24-9c96024c9c9a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:42.300940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30e119-4345-40a2-ad24-9c96024c9c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:42.301236 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30e119-4345-40a2-ad24-9c96024c9c9a None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:42.301906 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30e119-4345-40a2-ad24-9c96024c9c9a None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:42.302206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30e119-4345-40a2-ad24-9c96024c9c9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:42.303578 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30e119-4345-40a2-ad24-9c96024c9c9a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:42.307718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc738e-9864-423f-95da-2e8db274de97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:42.308374 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc738e-9864-423f-95da-2e8db274de97 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:42.308651 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc738e-9864-423f-95da-2e8db274de97 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:42.308922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc738e-9864-423f-95da-2e8db274de97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:42.328161 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 277/549] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 18:59:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:42.334622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30e119-4345-40a2-ad24-9c96024c9c9a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:42.335826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc738e-9864-423f-95da-2e8db274de97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:42.337338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc738e-9864-423f-95da-2e8db274de97 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:42.366610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-e7bc738e-9864-423f-95da-2e8db274de97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:42.366610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:42.366610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:42.366610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:42.366610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:42.366610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:42.366610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:42.366610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:42.366610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:42.366610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:42.366610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:42.366610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:42.366610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:42.366610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:42.366610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:42.366610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:42.368219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:42.369082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:42.369082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:42.369082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:42.371184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 273/550] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:42.377074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc738e-9864-423f-95da-2e8db274de97 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:42.482522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc738e-9864-423f-95da-2e8db274de97 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:42.484383 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:42.506357 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30e119-4345-40a2-ad24-9c96024c9c9a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:42.530330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed7888-4220-4c62-90ef-2c07f19f51e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:42.531396 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed7888-4220-4c62-90ef-2c07f19f51e6 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:42.531704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed7888-4220-4c62-90ef-2c07f19f51e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:42.531962 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed7888-4220-4c62-90ef-2c07f19f51e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:42.532594 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed7888-4220-4c62-90ef-2c07f19f51e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:42.532967 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed7888-4220-4c62-90ef-2c07f19f51e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:42.534423 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed7888-4220-4c62-90ef-2c07f19f51e6 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:42.553480 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64824dab-d6a4-4ab1-bf91-048647d931ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:42.554405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64824dab-d6a4-4ab1-bf91-048647d931ca None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:42.554884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64824dab-d6a4-4ab1-bf91-048647d931ca None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:42.555362 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64824dab-d6a4-4ab1-bf91-048647d931ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:42.561532 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 274/551] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:42.562434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed7888-4220-4c62-90ef-2c07f19f51e6 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:42.569322 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64824dab-d6a4-4ab1-bf91-048647d931ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:42.571970 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64824dab-d6a4-4ab1-bf91-048647d931ca None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:42.599637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-64824dab-d6a4-4ab1-bf91-048647d931ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:42.599637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:42.599637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:42.599637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:42.599637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:42.599637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:42.599637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:42.599637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:42.599637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:42.599637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:42.599637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:42.599637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:42.599637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:42.599637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:42.599637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:42.599637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:42.600706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:42.601694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:42.601694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:42.601694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:42.615197 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 278/552] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:42.616903 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64824dab-d6a4-4ab1-bf91-048647d931ca None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:42.663852 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:42.717164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed7888-4220-4c62-90ef-2c07f19f51e6 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:42.751424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615b845-fbbe-469e-a069-bf597357dd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:42.752051 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615b845-fbbe-469e-a069-bf597357dd04 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:42.752322 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615b845-fbbe-469e-a069-bf597357dd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:42.752588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8615b845-fbbe-469e-a069-bf597357dd04 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:42.760500 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615b845-fbbe-469e-a069-bf597357dd04 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:42.762592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8615b845-fbbe-469e-a069-bf597357dd04 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:42.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-8615b845-fbbe-469e-a069-bf597357dd04 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 18:59:42.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:42.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:42.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:42.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:42.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:42.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:42.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:42.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:42.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:42.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:42.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:42.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 18:59:42.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 18:59:42.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 18:59:42.774047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:42.774987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:42.775911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:42.775911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:42.775911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 18:59:42.775911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 18:59:42.775911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 18:59:42.775911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 18:59:42.775911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 18:59:42.775911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 18:59:42.775911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 18:59:42.775911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:42.775911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 275/553] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 18:59:42 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:42.777858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64824dab-d6a4-4ab1-bf91-048647d931ca None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:42.782284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615b845-fbbe-469e-a069-bf597357dd04 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:42.812261 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:42.814615 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5010d4-a84d-4b26-8823-f626cd7d2524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:42.815224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5010d4-a84d-4b26-8823-f626cd7d2524 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:42.815504 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5010d4-a84d-4b26-8823-f626cd7d2524 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:42.815775 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5010d4-a84d-4b26-8823-f626cd7d2524 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:42.816458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5010d4-a84d-4b26-8823-f626cd7d2524 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:42.816760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5010d4-a84d-4b26-8823-f626cd7d2524 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:42.818231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5010d4-a84d-4b26-8823-f626cd7d2524 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:42.847611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 279/554] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:42.849950 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5010d4-a84d-4b26-8823-f626cd7d2524 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:42.894391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8615b845-fbbe-469e-a069-bf597357dd04 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:42.955209 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852267fd-e077-46c8-80db-ab722506bfbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:42.956166 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852267fd-e077-46c8-80db-ab722506bfbc None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:42.957819 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852267fd-e077-46c8-80db-ab722506bfbc None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:42.958332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852267fd-e077-46c8-80db-ab722506bfbc None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:42.959259 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852267fd-e077-46c8-80db-ab722506bfbc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:42.959956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852267fd-e077-46c8-80db-ab722506bfbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:42.962507 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852267fd-e077-46c8-80db-ab722506bfbc None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:42.995313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 276/555] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 18:59:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:43.003719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852267fd-e077-46c8-80db-ab722506bfbc None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:43.040614 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:43.049964 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5010d4-a84d-4b26-8823-f626cd7d2524 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:43.126098 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3df4e-ca59-4d3c-8518-4c4a93d6d555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:43.128322 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3df4e-ca59-4d3c-8518-4c4a93d6d555 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:43.128860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3df4e-ca59-4d3c-8518-4c4a93d6d555 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:43.129381 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3df4e-ca59-4d3c-8518-4c4a93d6d555 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:43.146589 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3df4e-ca59-4d3c-8518-4c4a93d6d555 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:43.148376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3df4e-ca59-4d3c-8518-4c4a93d6d555 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:43.155679 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:43.164201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-0be3df4e-ca59-4d3c-8518-4c4a93d6d555 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 18:59:43.164201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:43.164201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:43.164201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:43.164201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:43.164201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:43.164201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:43.164201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:43.164201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:43.164201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:43.164201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:43.164201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:43.164201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 18:59:43.164201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 18:59:43.164201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 18:59:43.164201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:43.165871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:43.166788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:43.166788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:43.166788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 18:59:43.166788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 18:59:43.166788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 18:59:43.166788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 18:59:43.166788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 18:59:43.166788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 18:59:43.166788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 18:59:43.166788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:43.166788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 280/556] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:59:42 2022] GET /identity/v3/users/swift => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:43.167394 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac13f09-1704-4474-8bb9-91735dd39f0c None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:43.167394 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac13f09-1704-4474-8bb9-91735dd39f0c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:43.168481 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac13f09-1704-4474-8bb9-91735dd39f0c None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:43.169800 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 281/557] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:43.176857 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac13f09-1704-4474-8bb9-91735dd39f0c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:43.188821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852267fd-e077-46c8-80db-ab722506bfbc None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:43.242459 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbc3ff1-93fa-495e-83ad-be8d1b11d762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:43.243096 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc3ff1-93fa-495e-83ad-be8d1b11d762 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:43.243701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc3ff1-93fa-495e-83ad-be8d1b11d762 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:43.243997 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbc3ff1-93fa-495e-83ad-be8d1b11d762 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:43.257983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc3ff1-93fa-495e-83ad-be8d1b11d762 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:43.259781 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbc3ff1-93fa-495e-83ad-be8d1b11d762 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:43.276616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-0bbc3ff1-93fa-495e-83ad-be8d1b11d762 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:43.276616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:43.276616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:43.276616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:43.276616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:43.276616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:43.276616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:43.276616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:43.276616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:43.276616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:43.276616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:43.276616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:43.276616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:43.276616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:43.276616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:43.276616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:43.277871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:43.279119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:43.279119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:43.279119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:43.279119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 277/558] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:43.279402 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5491846-6f8e-4683-a83b-477a5a7cabf9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:43.279518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5491846-6f8e-4683-a83b-477a5a7cabf9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:43.279792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5491846-6f8e-4683-a83b-477a5a7cabf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:43.299023 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f5491846-6f8e-4683-a83b-477a5a7cabf9 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:43.300211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f5491846-6f8e-4683-a83b-477a5a7cabf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:43.301520 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac13f09-1704-4474-8bb9-91735dd39f0c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:43.337656 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dc433b-bad3-472f-ac3f-cd5de1c0c40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:43.338261 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc433b-bad3-472f-ac3f-cd5de1c0c40c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:43.338533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc433b-bad3-472f-ac3f-cd5de1c0c40c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:43.338796 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dc433b-bad3-472f-ac3f-cd5de1c0c40c None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:43.339536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc433b-bad3-472f-ac3f-cd5de1c0c40c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:43.339847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc433b-bad3-472f-ac3f-cd5de1c0c40c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:43.341292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dc433b-bad3-472f-ac3f-cd5de1c0c40c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:43.359287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 282/559] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 31 18:59:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:43.360563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ebf1a-2e97-4446-aa84-b981a2685043 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:43.360844 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ebf1a-2e97-4446-aa84-b981a2685043 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:43.361112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0ebf1a-2e97-4446-aa84-b981a2685043 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:43.366404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 283/560] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:43.367033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ebf1a-2e97-4446-aa84-b981a2685043 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:43.376537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 278/561] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:43.377279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5491846-6f8e-4683-a83b-477a5a7cabf9 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:43.387409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:43.442283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:43.484614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5491846-6f8e-4683-a83b-477a5a7cabf9 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:43.503839 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0ebf1a-2e97-4446-aa84-b981a2685043 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:43.521385 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448373e-ab25-40db-af2f-03eb60c6afca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:43.521924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448373e-ab25-40db-af2f-03eb60c6afca None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:43.522201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448373e-ab25-40db-af2f-03eb60c6afca None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:43.522474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2448373e-ab25-40db-af2f-03eb60c6afca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:43.550033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942a5696-cbac-46fe-bd26-381d5832eab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:43.551875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a5696-cbac-46fe-bd26-381d5832eab7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:43.552364 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a5696-cbac-46fe-bd26-381d5832eab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:43.552837 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942a5696-cbac-46fe-bd26-381d5832eab7 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:43.553814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a5696-cbac-46fe-bd26-381d5832eab7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:43.554615 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a5696-cbac-46fe-bd26-381d5832eab7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:43.556606 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942a5696-cbac-46fe-bd26-381d5832eab7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:43.572301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 284/562] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:43.574321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa9ce3-dfe0-45d4-857b-d38329de685b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:43.575074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa9ce3-dfe0-45d4-857b-d38329de685b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:43.575886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa9ce3-dfe0-45d4-857b-d38329de685b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:43.597684 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-25fa9ce3-dfe0-45d4-857b-d38329de685b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:43.598999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-25fa9ce3-dfe0-45d4-857b-d38329de685b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:43.599741 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448373e-ab25-40db-af2f-03eb60c6afca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:43.601706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2448373e-ab25-40db-af2f-03eb60c6afca None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:43.673717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:43.682781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 285/563] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:43.682781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa9ce3-dfe0-45d4-857b-d38329de685b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:43.682781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:43.724608 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 279/564] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:43.725462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448373e-ab25-40db-af2f-03eb60c6afca None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:43.819265 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa9ce3-dfe0-45d4-857b-d38329de685b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:43.846595 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2448373e-ab25-40db-af2f-03eb60c6afca None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:43.863268 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01335998-2a60-4349-b2ca-b88a635586b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:43.864166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01335998-2a60-4349-b2ca-b88a635586b9 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:43.864516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01335998-2a60-4349-b2ca-b88a635586b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:43.865123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01335998-2a60-4349-b2ca-b88a635586b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:43.874092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01335998-2a60-4349-b2ca-b88a635586b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:43.876241 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01335998-2a60-4349-b2ca-b88a635586b9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:43.882430 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf6a71-6641-42af-b8ce-90cbf178ec15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:43.883053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf6a71-6641-42af-b8ce-90cbf178ec15 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:43.883374 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf6a71-6641-42af-b8ce-90cbf178ec15 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:43.883642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bf6a71-6641-42af-b8ce-90cbf178ec15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:43.889807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-01335998-2a60-4349-b2ca-b88a635586b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:43.889807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:43.889807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:43.889807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:43.889807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:43.889807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:43.889807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:43.889807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:43.889807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:43.889807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:43.889807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:43.889807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:43.889807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 18:59:43.889807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 18:59:43.889807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:43.890677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:43.891553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 18:59:43.891553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 18:59:43.891553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 18:59:43.891553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 18:59:43.891553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 18:59:43.891553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 18:59:43.891553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:43.891553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:43.891553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 286/565] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:43.892041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01335998-2a60-4349-b2ca-b88a635586b9 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:43.900873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bf6a71-6641-42af-b8ce-90cbf178ec15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:43.906806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-c9bf6a71-6641-42af-b8ce-90cbf178ec15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:43.906806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:43.906806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:43.906806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:43.906806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:43.906806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:43.906806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:43.906806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:43.906806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:43.906806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:43.906806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:43.906806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:43.906806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:43.906806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:43.906806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:43.906806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:43.908888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:43.908888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:43.908888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:43.908888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:43.908888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:43.908888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:43.908888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:43.908888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:43.908888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:43.908888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:43.908888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:43.908888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:43.908888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:43.908888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:43.908888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:43.908888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:43.910924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:43.910924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:43.910924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:43.910924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:43.910924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:43.910924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:43.910924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:43.910924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:43.910924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:43.910924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:43.910924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:43.910924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:43.917312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 280/566] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:43.917312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf6a71-6641-42af-b8ce-90cbf178ec15 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:43.943013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:43.965395 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:44.005034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01335998-2a60-4349-b2ca-b88a635586b9 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:44.018382 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bf6a71-6641-42af-b8ce-90cbf178ec15 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:44.038972 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198bd3a-aba8-4275-aa29-0d81e4641262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:44.039588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198bd3a-aba8-4275-aa29-0d81e4641262 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:44.039858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198bd3a-aba8-4275-aa29-0d81e4641262 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:44.040125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6198bd3a-aba8-4275-aa29-0d81e4641262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:44.050373 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198bd3a-aba8-4275-aa29-0d81e4641262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:44.051777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6198bd3a-aba8-4275-aa29-0d81e4641262 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:44.053597 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce14615c-28c2-42e9-aa0e-70e8da49e885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:44.054190 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14615c-28c2-42e9-aa0e-70e8da49e885 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:44.054463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14615c-28c2-42e9-aa0e-70e8da49e885 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:44.054732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14615c-28c2-42e9-aa0e-70e8da49e885 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:44.055404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14615c-28c2-42e9-aa0e-70e8da49e885 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:44.055701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14615c-28c2-42e9-aa0e-70e8da49e885 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:44.057117 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce14615c-28c2-42e9-aa0e-70e8da49e885 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:44.065321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-6198bd3a-aba8-4275-aa29-0d81e4641262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:44.065321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:44.065321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:44.065321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:44.065321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:44.065321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:44.065321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:44.065321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:44.065321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:44.065321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:44.065321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:44.065321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:44.065321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:44.065321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:44.065321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:44.065321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:44.066224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:44.072079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:44.072079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:44.072079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:44.072079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 287/567] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:44.072079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198bd3a-aba8-4275-aa29-0d81e4641262 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:44.075220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 281/568] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 18:59:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:44.076424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f7599-cf68-4860-a631-2a02925b7816 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:44.076708 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f7599-cf68-4860-a631-2a02925b7816 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:44.076979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f7599-cf68-4860-a631-2a02925b7816 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:44.081305 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 282/569] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:44.082317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f7599-cf68-4860-a631-2a02925b7816 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:44.164482 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:44.166794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6198bd3a-aba8-4275-aa29-0d81e4641262 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:44.180517 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:44.183503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f7599-cf68-4860-a631-2a02925b7816 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:44.201860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80673d2-39e2-49e1-ab47-b3a613bbb9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:44.202462 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80673d2-39e2-49e1-ab47-b3a613bbb9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:44.202740 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80673d2-39e2-49e1-ab47-b3a613bbb9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:44.203010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80673d2-39e2-49e1-ab47-b3a613bbb9f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:44.203666 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80673d2-39e2-49e1-ab47-b3a613bbb9f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:44.203966 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80673d2-39e2-49e1-ab47-b3a613bbb9f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:44.205362 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80673d2-39e2-49e1-ab47-b3a613bbb9f0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:44.216013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 288/570] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 18:59:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:44.216823 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80673d2-39e2-49e1-ab47-b3a613bbb9f0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:44.222712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4242c-52b2-4435-a53d-8fa3074bcc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:44.223341 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4242c-52b2-4435-a53d-8fa3074bcc57 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:44.223620 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4242c-52b2-4435-a53d-8fa3074bcc57 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:44.223892 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f4242c-52b2-4435-a53d-8fa3074bcc57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:44.234155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f4242c-52b2-4435-a53d-8fa3074bcc57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:44.238922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-90f4242c-52b2-4435-a53d-8fa3074bcc57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:44.238922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:44.238922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:44.238922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:44.238922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:44.238922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:44.238922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:44.238922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:44.238922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:44.238922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:44.238922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:44.238922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:44.238922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:44.238922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:44.238922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:44.238922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:44.239939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:44.239939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:44.239939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:44.239939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:44.239939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:44.239939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:44.239939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:44.239939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:44.239939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:44.239939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:44.239939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:44.239939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:44.239939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:44.239939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:44.239939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:44.239939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:44.240782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:44.240782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:44.240782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:44.240782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:44.240782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:44.240782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:44.240782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:44.240782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:44.240782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:44.240782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:44.240782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:44.240782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:44.240782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 283/571] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:44.241522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4242c-52b2-4435-a53d-8fa3074bcc57 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:44.315383 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80673d2-39e2-49e1-ab47-b3a613bbb9f0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:44.341358 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f4242c-52b2-4435-a53d-8fa3074bcc57 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:44.348594 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f733eb5a-8908-4d4b-a2f3-c9594c6970ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:44.349207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733eb5a-8908-4d4b-a2f3-c9594c6970ee None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:44.349495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733eb5a-8908-4d4b-a2f3-c9594c6970ee None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:44.349767 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f733eb5a-8908-4d4b-a2f3-c9594c6970ee None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:44.350393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733eb5a-8908-4d4b-a2f3-c9594c6970ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:44.350692 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733eb5a-8908-4d4b-a2f3-c9594c6970ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:44.352042 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f733eb5a-8908-4d4b-a2f3-c9594c6970ee None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:44.363762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 289/572] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:44.364995 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566bbba-0ca7-414d-88a0-7823eec56b9c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:44.365311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566bbba-0ca7-414d-88a0-7823eec56b9c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:44.365584 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566bbba-0ca7-414d-88a0-7823eec56b9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:44.380901 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d38c9-90c6-470b-b9e3-f3662f5a6134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:44.381774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d38c9-90c6-470b-b9e3-f3662f5a6134 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:44.382206 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d38c9-90c6-470b-b9e3-f3662f5a6134 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:44.382647 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d38c9-90c6-470b-b9e3-f3662f5a6134 None admin] PATH_INFO: `/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/673a72ac9a9847e3812007b66ac7a5ea/roles/6e93fc836fe74c9daa11c7b2848a2922` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:44.383797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-1566bbba-0ca7-414d-88a0-7823eec56b9c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:44.384978 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1566bbba-0ca7-414d-88a0-7823eec56b9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:44.395610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:44.398968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d38c9-90c6-470b-b9e3-f3662f5a6134 None admin] RBAC: Authorizing `identity:create_grant(project_id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user_id=673a72ac9a9847e3812007b66ac7a5ea, role_id=6e93fc836fe74c9daa11c7b2848a2922)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:44.400693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891d38c9-90c6-470b-b9e3-f3662f5a6134 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:44.414538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:44.425256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 284/573] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 18:59:44 2022] PUT /identity/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/673a72ac9a9847e3812007b66ac7a5ea/roles/6e93fc836fe74c9daa11c7b2848a2922 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 18:59:44.425256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d38c9-90c6-470b-b9e3-f3662f5a6134 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:44.470986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 290/574] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:44.472179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566bbba-0ca7-414d-88a0-7823eec56b9c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:44.541736 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d38c9-90c6-470b-b9e3-f3662f5a6134 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:44.591682 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25caa8-cf56-44c7-8001-fbd4765a6a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:44.592316 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25caa8-cf56-44c7-8001-fbd4765a6a62 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:44.592700 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25caa8-cf56-44c7-8001-fbd4765a6a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:44.592981 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25caa8-cf56-44c7-8001-fbd4765a6a62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:44.604693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25caa8-cf56-44c7-8001-fbd4765a6a62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:44.606666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25caa8-cf56-44c7-8001-fbd4765a6a62 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:44.606843 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566bbba-0ca7-414d-88a0-7823eec56b9c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:44.627071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-2a25caa8-cf56-44c7-8001-fbd4765a6a62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:44.627071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:44.627071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:44.627071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:44.627071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:44.627071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:44.627071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:44.627071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:44.627071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:44.627071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:44.627071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:44.627071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:44.627071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:44.627071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:44.627071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:44.627071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:44.628046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:44.628918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:44.628918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:44.628918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:44.628918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 285/575] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:44.629602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25caa8-cf56-44c7-8001-fbd4765a6a62 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:44.648296 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2bd83a-2463-473d-a38c-1092b45344df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:44.648903 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bd83a-2463-473d-a38c-1092b45344df None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:44.649181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bd83a-2463-473d-a38c-1092b45344df None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:44.649477 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2bd83a-2463-473d-a38c-1092b45344df None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:44.650130 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2bd83a-2463-473d-a38c-1092b45344df None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:44.650426 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2bd83a-2463-473d-a38c-1092b45344df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:44.651784 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2bd83a-2463-473d-a38c-1092b45344df None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:44.660378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 291/576] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 18:59:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:44.661214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2bd83a-2463-473d-a38c-1092b45344df None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:44.681360 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:44.696100 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:44.744738 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25caa8-cf56-44c7-8001-fbd4765a6a62 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:44.771119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2bd83a-2463-473d-a38c-1092b45344df None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:44.783426 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ad298-c48e-44b9-b561-834476d149ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:44.784057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ad298-c48e-44b9-b561-834476d149ea None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:44.784334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ad298-c48e-44b9-b561-834476d149ea None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:44.784603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ad298-c48e-44b9-b561-834476d149ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:44.791486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ad298-c48e-44b9-b561-834476d149ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:44.793477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ad298-c48e-44b9-b561-834476d149ea None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:44.804699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-835ad298-c48e-44b9-b561-834476d149ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:44.804699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:44.804699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:44.804699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:44.804699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:44.804699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:44.804699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:44.804699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:44.804699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:44.804699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:44.804699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:44.804699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:44.804699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 18:59:44.804699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 18:59:44.804699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:44.805687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:44.807092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 18:59:44.807092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 18:59:44.807092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 18:59:44.807092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 18:59:44.807092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 18:59:44.807092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 18:59:44.807092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:44.807092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:44.807092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 286/577] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:44.807579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd216f6-9ede-4844-9125-ac5950995046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:44.807579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd216f6-9ede-4844-9125-ac5950995046 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:44.807579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd216f6-9ede-4844-9125-ac5950995046 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:44.807579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd216f6-9ede-4844-9125-ac5950995046 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:44.807864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd216f6-9ede-4844-9125-ac5950995046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:44.808162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd216f6-9ede-4844-9125-ac5950995046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:44.810101 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd216f6-9ede-4844-9125-ac5950995046 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:44.811673 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ad298-c48e-44b9-b561-834476d149ea None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:44.826858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 292/578] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:44.826858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd216f6-9ede-4844-9125-ac5950995046 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:44.916880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:44.920599 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ad298-c48e-44b9-b561-834476d149ea None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:44.923185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:44.932222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd216f6-9ede-4844-9125-ac5950995046 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:44.957214 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0453ef8-e44b-482b-a324-4464156385df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:44.957845 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0453ef8-e44b-482b-a324-4464156385df None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:44.958127 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0453ef8-e44b-482b-a324-4464156385df None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:44.958403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0453ef8-e44b-482b-a324-4464156385df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:44.968342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1da5b9-fce1-40c9-99a5-3a45b2edd277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:44.968964 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1da5b9-fce1-40c9-99a5-3a45b2edd277 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:44.969275 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1da5b9-fce1-40c9-99a5-3a45b2edd277 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:44.969582 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1da5b9-fce1-40c9-99a5-3a45b2edd277 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:44.969703 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0453ef8-e44b-482b-a324-4464156385df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:44.971006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0453ef8-e44b-482b-a324-4464156385df None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:44.980086 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1da5b9-fce1-40c9-99a5-3a45b2edd277 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:44.985399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-1f1da5b9-fce1-40c9-99a5-3a45b2edd277 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:44.985399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:44.985399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:44.985399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:44.985399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:44.985399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:44.985399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:44.985399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:44.985399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:44.985399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:44.985399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:44.985399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:44.985399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:44.985399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:44.985399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:44.985399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:44.986522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:44.986522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:44.986522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:44.986522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:44.986522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:44.986522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:44.986522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:44.986522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:44.986522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:44.986522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:44.986522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:44.986522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:44.986522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:44.986522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:44.986522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:44.986522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:44.992884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:44.992884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:44.992884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:44.992884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:44.992884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:44.992884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:44.992884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:44.992884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:44.992884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:44.992884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:44.992884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:44.992884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:44.992884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 293/579] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:44.993861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-b0453ef8-e44b-482b-a324-4464156385df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:44.993861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:44.993861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:44.993861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:44.993861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:44.993861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:44.993861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:44.993861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:44.993861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:44.993861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:44.993861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:44.993861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:44.993861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:44.993861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:44.993861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:44.993861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:44.994914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:44.996082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:44.996082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:44.996082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:44.996286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1da5b9-fce1-40c9-99a5-3a45b2edd277 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:44.996377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 287/580] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:44.996377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0453ef8-e44b-482b-a324-4464156385df None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:45.100887 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0453ef8-e44b-482b-a324-4464156385df None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:45.107141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1da5b9-fce1-40c9-99a5-3a45b2edd277 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:45.136790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101aef69-ee62-42b8-812a-20556596a651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:45.137444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101aef69-ee62-42b8-812a-20556596a651 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:45.137736 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101aef69-ee62-42b8-812a-20556596a651 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:45.138100 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101aef69-ee62-42b8-812a-20556596a651 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:45.143125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406f71d-d163-430d-a483-f37264fdb970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:45.143723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406f71d-d163-430d-a483-f37264fdb970 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:45.144008 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406f71d-d163-430d-a483-f37264fdb970 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:45.144270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1406f71d-d163-430d-a483-f37264fdb970 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:45.144951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406f71d-d163-430d-a483-f37264fdb970 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:45.145280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406f71d-d163-430d-a483-f37264fdb970 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:45.145818 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101aef69-ee62-42b8-812a-20556596a651 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:45.146713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1406f71d-d163-430d-a483-f37264fdb970 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:45.147242 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101aef69-ee62-42b8-812a-20556596a651 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:45.163761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 294/581] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 18:59:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:45.163948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-101aef69-ee62-42b8-812a-20556596a651 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 18:59:45.163948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:45.163948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:45.163948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:45.163948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:45.163948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:45.163948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:45.163948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:45.163948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:45.163948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:45.163948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:45.163948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:45.163948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 18:59:45.163948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 18:59:45.163948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 18:59:45.163948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:45.164965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:45.165958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:45.165958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:45.165958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 18:59:45.165958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 18:59:45.165958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 18:59:45.165958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 18:59:45.165958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 18:59:45.165958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 18:59:45.165958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 18:59:45.165958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:45.166505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406f71d-d163-430d-a483-f37264fdb970 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:45.166575 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 288/582] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:44 2022] GET /identity/v3/users/glance => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:45.166575 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101aef69-ee62-42b8-812a-20556596a651 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:45.178956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:45.187483 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:45.273865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1406f71d-d163-430d-a483-f37264fdb970 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:45.277915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101aef69-ee62-42b8-812a-20556596a651 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:45.310153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce12bc3-73be-4b11-b950-13e517385557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:45.310754 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce12bc3-73be-4b11-b950-13e517385557 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:45.311039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce12bc3-73be-4b11-b950-13e517385557 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:45.311309 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce12bc3-73be-4b11-b950-13e517385557 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:45.311961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce12bc3-73be-4b11-b950-13e517385557 None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:45.312260 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce12bc3-73be-4b11-b950-13e517385557 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:45.313646 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce12bc3-73be-4b11-b950-13e517385557 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:45.315246 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec676d62-2cdd-4af6-ae67-7a08d2d14865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:45.316129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec676d62-2cdd-4af6-ae67-7a08d2d14865 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:45.316582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec676d62-2cdd-4af6-ae67-7a08d2d14865 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:45.317038 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec676d62-2cdd-4af6-ae67-7a08d2d14865 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:45.317886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec676d62-2cdd-4af6-ae67-7a08d2d14865 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:45.318358 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec676d62-2cdd-4af6-ae67-7a08d2d14865 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:45.319903 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec676d62-2cdd-4af6-ae67-7a08d2d14865 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:45.321982 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 295/583] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 18:59:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:45.322924 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce12bc3-73be-4b11-b950-13e517385557 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:45.337355 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 289/584] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:45.338275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec676d62-2cdd-4af6-ae67-7a08d2d14865 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:45.412591 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:45.419897 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:45.446293 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec676d62-2cdd-4af6-ae67-7a08d2d14865 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:45.447952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce12bc3-73be-4b11-b950-13e517385557 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:45.480989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5cbab0-2cf4-41ab-94d0-685ba5ab68a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:45.481645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cbab0-2cf4-41ab-94d0-685ba5ab68a2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:45.481923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cbab0-2cf4-41ab-94d0-685ba5ab68a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:45.482195 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5cbab0-2cf4-41ab-94d0-685ba5ab68a2 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:45.482923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5cbab0-2cf4-41ab-94d0-685ba5ab68a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:45.483234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5cbab0-2cf4-41ab-94d0-685ba5ab68a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:45.484655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5cbab0-2cf4-41ab-94d0-685ba5ab68a2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:45.485862 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806dc9c-d60b-4757-8c47-d79a2641b1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:45.486699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806dc9c-d60b-4757-8c47-d79a2641b1e4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:45.487237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806dc9c-d60b-4757-8c47-d79a2641b1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:45.487508 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6806dc9c-d60b-4757-8c47-d79a2641b1e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:45.488503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806dc9c-d60b-4757-8c47-d79a2641b1e4 None admin] RBAC: Adding query filter params (role.id=6e93fc836fe74c9daa11c7b2848a2922, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=586715d11b924e2cb6f44030efd77619) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:45.488814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806dc9c-d60b-4757-8c47-d79a2641b1e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:45.490242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6806dc9c-d60b-4757-8c47-d79a2641b1e4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:45.496114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 296/585] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:59:45 2022] GET /identity/v3/role_assignments?user.id=586715d11b924e2cb6f44030efd77619&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=6e93fc836fe74c9daa11c7b2848a2922 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:45.496709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806dc9c-d60b-4757-8c47-d79a2641b1e4 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:45.509260 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 290/586] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 31 18:59:45 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:45.509260 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5cbab0-2cf4-41ab-94d0-685ba5ab68a2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:45.627447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5cbab0-2cf4-41ab-94d0-685ba5ab68a2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:45.628520 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6806dc9c-d60b-4757-8c47-d79a2641b1e4 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:45.669925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0a257-d266-4cbb-bec9-c0902f13fd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:45.670281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecddfcd-bc93-4bd2-80a8-c14d85c0541b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:45.670556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0a257-d266-4cbb-bec9-c0902f13fd92 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:45.670836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0a257-d266-4cbb-bec9-c0902f13fd92 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:45.671108 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f0a257-d266-4cbb-bec9-c0902f13fd92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:45.671633 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecddfcd-bc93-4bd2-80a8-c14d85c0541b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:45.672099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecddfcd-bc93-4bd2-80a8-c14d85c0541b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:45.672538 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecddfcd-bc93-4bd2-80a8-c14d85c0541b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:45.673333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:45.682536 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:45.684269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecddfcd-bc93-4bd2-80a8-c14d85c0541b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:45.685880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f0a257-d266-4cbb-bec9-c0902f13fd92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:45.686025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecddfcd-bc93-4bd2-80a8-c14d85c0541b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:45.688349 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f0a257-d266-4cbb-bec9-c0902f13fd92 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:45.702503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-5ecddfcd-bc93-4bd2-80a8-c14d85c0541b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 18:59:45.702503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:45.702503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:45.702503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:45.702503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:45.702503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:45.702503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:45.702503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:45.702503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:45.702503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:45.702503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:45.702503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:45.702503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 18:59:45.702503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 18:59:45.702503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 18:59:45.702503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:45.703473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:45.704447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:45.704447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:45.704447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 18:59:45.704447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 18:59:45.704447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 18:59:45.704447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 18:59:45.704447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 18:59:45.704447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 18:59:45.704447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 18:59:45.704447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:45.704447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 291/587] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:45.705807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecddfcd-bc93-4bd2-80a8-c14d85c0541b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:45.708271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-26f0a257-d266-4cbb-bec9-c0902f13fd92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:45.708271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:45.708271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:45.708271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:45.708271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:45.708271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:45.708271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:45.708271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:45.708271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:45.708271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:45.708271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:45.708271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:45.708271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:45.708271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:45.708271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:45.708271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:45.709208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:45.710198 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:45.710198 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:45.710198 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:45.712404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 297/588] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:45.713385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0a257-d266-4cbb-bec9-c0902f13fd92 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:45.829359 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecddfcd-bc93-4bd2-80a8-c14d85c0541b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:45.844626 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f0a257-d266-4cbb-bec9-c0902f13fd92 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:45.866522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214aa0d6-7b3d-4b17-8659-95cbe25682c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:45.867445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214aa0d6-7b3d-4b17-8659-95cbe25682c3 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:45.867929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214aa0d6-7b3d-4b17-8659-95cbe25682c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:45.868385 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214aa0d6-7b3d-4b17-8659-95cbe25682c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:45.884193 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214aa0d6-7b3d-4b17-8659-95cbe25682c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:45.885861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214aa0d6-7b3d-4b17-8659-95cbe25682c3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:45.886663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3cdda-04df-495c-8d56-0c949cd05c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:45.887880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3cdda-04df-495c-8d56-0c949cd05c5e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:45.888254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3cdda-04df-495c-8d56-0c949cd05c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:45.888631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b3cdda-04df-495c-8d56-0c949cd05c5e None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:45.889720 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3cdda-04df-495c-8d56-0c949cd05c5e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:45.890384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3cdda-04df-495c-8d56-0c949cd05c5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:45.891889 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b3cdda-04df-495c-8d56-0c949cd05c5e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:45.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-214aa0d6-7b3d-4b17-8659-95cbe25682c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:45.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:45.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:45.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:45.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:45.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:45.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:45.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:45.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:45.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:45.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:45.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:45.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:45.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:45.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:45.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:45.906462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:45.907534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:45.907534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:45.907534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:45.913927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 292/589] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:45.914217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214aa0d6-7b3d-4b17-8659-95cbe25682c3 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:45.918607 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 298/590] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 31 18:59:45 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:45.919502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3cdda-04df-495c-8d56-0c949cd05c5e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:45.972828 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:45.982891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:46.032457 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b3cdda-04df-495c-8d56-0c949cd05c5e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:46.044174 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214aa0d6-7b3d-4b17-8659-95cbe25682c3 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:46.068969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79129a71-d958-4c5f-a287-e2b2d9f91cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:46.069961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79129a71-d958-4c5f-a287-e2b2d9f91cee None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:46.070444 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79129a71-d958-4c5f-a287-e2b2d9f91cee None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:46.070913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79129a71-d958-4c5f-a287-e2b2d9f91cee None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:46.071772 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79129a71-d958-4c5f-a287-e2b2d9f91cee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:46.072272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79129a71-d958-4c5f-a287-e2b2d9f91cee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:46.073863 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79129a71-d958-4c5f-a287-e2b2d9f91cee None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:46.081885 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbdfc3-caf2-406e-ae8b-e89a3ece4b59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:46.082492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbdfc3-caf2-406e-ae8b-e89a3ece4b59 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:46.082770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbdfc3-caf2-406e-ae8b-e89a3ece4b59 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:46.083044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bbdfc3-caf2-406e-ae8b-e89a3ece4b59 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:46.083681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbdfc3-caf2-406e-ae8b-e89a3ece4b59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:46.084008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbdfc3-caf2-406e-ae8b-e89a3ece4b59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:46.085557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bbdfc3-caf2-406e-ae8b-e89a3ece4b59 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:46.092292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 299/591] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:46.093410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79129a71-d958-4c5f-a287-e2b2d9f91cee None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:46.102931 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 293/592] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:46.103791 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbdfc3-caf2-406e-ae8b-e89a3ece4b59 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:46.199623 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79129a71-d958-4c5f-a287-e2b2d9f91cee None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:46.202221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:46.218963 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bbdfc3-caf2-406e-ae8b-e89a3ece4b59 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:46.227337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:46.239912 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926966e-18cb-4c3d-8090-b05dc9e59363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:46.240619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926966e-18cb-4c3d-8090-b05dc9e59363 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:46.241001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926966e-18cb-4c3d-8090-b05dc9e59363 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:46.241400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3926966e-18cb-4c3d-8090-b05dc9e59363 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:46.252646 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926966e-18cb-4c3d-8090-b05dc9e59363 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:46.254225 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3926966e-18cb-4c3d-8090-b05dc9e59363 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:46.259033 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480f3e3-bb0c-4b29-bcb3-5bf9a9bc708a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:46.259854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480f3e3-bb0c-4b29-bcb3-5bf9a9bc708a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:46.260280 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480f3e3-bb0c-4b29-bcb3-5bf9a9bc708a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:46.260702 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480f3e3-bb0c-4b29-bcb3-5bf9a9bc708a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:46.268521 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480f3e3-bb0c-4b29-bcb3-5bf9a9bc708a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:46.269522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-3926966e-18cb-4c3d-8090-b05dc9e59363 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:46.269522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:46.269522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:46.269522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:46.269522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:46.269522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:46.269522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:46.269522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:46.269522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:46.269522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:46.269522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:46.269522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:46.269522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:46.269522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:46.269522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:46.269522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:46.270470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:46.271358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:46.271358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:46.271358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:46.271358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 300/593] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:46.271636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926966e-18cb-4c3d-8090-b05dc9e59363 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:46.278039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480f3e3-bb0c-4b29-bcb3-5bf9a9bc708a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:46.291011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-3480f3e3-bb0c-4b29-bcb3-5bf9a9bc708a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:46.291011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:46.291011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:46.291011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:46.291011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:46.291011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:46.291011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:46.291011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:46.291011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:46.291011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:46.291011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:46.291011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:46.291011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 18:59:46.291011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 18:59:46.291011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:46.291934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:46.292870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 18:59:46.292870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 18:59:46.292870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 18:59:46.292870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 18:59:46.292870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 18:59:46.292870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 18:59:46.292870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:46.292870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:46.295274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 294/594] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:46.296231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480f3e3-bb0c-4b29-bcb3-5bf9a9bc708a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:46.381121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3926966e-18cb-4c3d-8090-b05dc9e59363 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:46.406596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480f3e3-bb0c-4b29-bcb3-5bf9a9bc708a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:46.416253 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c1481-63a7-490d-b7bc-2f52eec9b3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:46.416882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c1481-63a7-490d-b7bc-2f52eec9b3c1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:46.417162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c1481-63a7-490d-b7bc-2f52eec9b3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:46.417460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340c1481-63a7-490d-b7bc-2f52eec9b3c1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:46.425027 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c1481-63a7-490d-b7bc-2f52eec9b3c1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:46.426489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340c1481-63a7-490d-b7bc-2f52eec9b3c1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:46.437732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-340c1481-63a7-490d-b7bc-2f52eec9b3c1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 18:59:46.437732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:46.437732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:46.437732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:46.437732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:46.437732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:46.437732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:46.437732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:46.437732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:46.437732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:46.437732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:46.437732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:46.437732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 18:59:46.437732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 18:59:46.437732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 18:59:46.437732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:46.438643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:46.439555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:46.439555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:46.439555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 18:59:46.439555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 18:59:46.439555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 18:59:46.439555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 18:59:46.439555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 18:59:46.439555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 18:59:46.439555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 18:59:46.439555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:46.439555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 301/595] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:46 2022] GET /identity/v3/users/neutron => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:46.440125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c1481-63a7-490d-b7bc-2f52eec9b3c1 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:46.447319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5d600-9497-4d77-9c19-24db102c3c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:46.447959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5d600-9497-4d77-9c19-24db102c3c0d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:46.448129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5d600-9497-4d77-9c19-24db102c3c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:46.448390 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5d600-9497-4d77-9c19-24db102c3c0d None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:46.449783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5d600-9497-4d77-9c19-24db102c3c0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:46.450099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5d600-9497-4d77-9c19-24db102c3c0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:46.451504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5d600-9497-4d77-9c19-24db102c3c0d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:46.461145 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:46.463666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 295/596] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:46.465257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a97e-d3e7-4f03-9f74-3a5012d7597f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:46.465545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a97e-d3e7-4f03-9f74-3a5012d7597f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:46.465819 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668a97e-d3e7-4f03-9f74-3a5012d7597f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:46.467269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 296/597] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:46.467972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668a97e-d3e7-4f03-9f74-3a5012d7597f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:46.494481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:46.562394 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340c1481-63a7-490d-b7bc-2f52eec9b3c1 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:46.577518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668a97e-d3e7-4f03-9f74-3a5012d7597f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:46.600299 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a6660-606a-4d95-8e49-ac653444b0af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:46.600903 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a6660-606a-4d95-8e49-ac653444b0af None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:46.601180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a6660-606a-4d95-8e49-ac653444b0af None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:46.601470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a6660-606a-4d95-8e49-ac653444b0af None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:46.602154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a6660-606a-4d95-8e49-ac653444b0af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:46.602451 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a6660-606a-4d95-8e49-ac653444b0af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:46.603879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a6660-606a-4d95-8e49-ac653444b0af None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:46.613022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dd8b0d-8d62-4a7c-ba6b-3373cbddd830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:46.613658 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd8b0d-8d62-4a7c-ba6b-3373cbddd830 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:46.613930 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd8b0d-8d62-4a7c-ba6b-3373cbddd830 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:46.614204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dd8b0d-8d62-4a7c-ba6b-3373cbddd830 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:46.614925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd8b0d-8d62-4a7c-ba6b-3373cbddd830 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:46.615229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd8b0d-8d62-4a7c-ba6b-3373cbddd830 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:46.616657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dd8b0d-8d62-4a7c-ba6b-3373cbddd830 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:46.622635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 302/598] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 18:59:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:46.622635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a6660-606a-4d95-8e49-ac653444b0af None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:46.641302 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 297/599] 10.222.0.65 () {56 vars in 1139 bytes} [Wed Aug 31 18:59:46 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:46.642846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68eb6f-3c50-4e1a-8d1d-a824c1b13c9f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:46.643311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68eb6f-3c50-4e1a-8d1d-a824c1b13c9f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:46.643761 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68eb6f-3c50-4e1a-8d1d-a824c1b13c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:46.664059 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-7e68eb6f-3c50-4e1a-8d1d-a824c1b13c9f None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:46.665317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7e68eb6f-3c50-4e1a-8d1d-a824c1b13c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:46.710606 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:46.735513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a6660-606a-4d95-8e49-ac653444b0af None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:46.741001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 298/600] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:46.741731 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68eb6f-3c50-4e1a-8d1d-a824c1b13c9f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:46.757484 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:46.789002 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00497e4a-360a-4d2c-b673-6d46bf38a984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:46.790172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00497e4a-360a-4d2c-b673-6d46bf38a984 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:46.790972 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00497e4a-360a-4d2c-b673-6d46bf38a984 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:46.791565 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00497e4a-360a-4d2c-b673-6d46bf38a984 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:46.800100 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00497e4a-360a-4d2c-b673-6d46bf38a984 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:46.802297 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00497e4a-360a-4d2c-b673-6d46bf38a984 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:46.817634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-00497e4a-360a-4d2c-b673-6d46bf38a984 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:46.817634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:46.817634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:46.817634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:46.817634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:46.817634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:46.817634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:46.817634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:46.817634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:46.817634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:46.817634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:46.817634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:46.817634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 18:59:46.817634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 18:59:46.817634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:46.818509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:46.819385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 18:59:46.819385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 18:59:46.819385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 18:59:46.819385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 18:59:46.819385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 18:59:46.819385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 18:59:46.819385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:46.819385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:46.821690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 303/601] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:46.822987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00497e4a-360a-4d2c-b673-6d46bf38a984 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:46.847408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68eb6f-3c50-4e1a-8d1d-a824c1b13c9f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:46.884247 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ae29a-bc0b-4335-8f4c-3a2b89ce6856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:46.884796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ae29a-bc0b-4335-8f4c-3a2b89ce6856 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:46.885179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ae29a-bc0b-4335-8f4c-3a2b89ce6856 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:46.885609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6ae29a-bc0b-4335-8f4c-3a2b89ce6856 None admin] PATH_INFO: `/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/1863336fecad4987acdfd0b3f7b2f70f/roles/6e93fc836fe74c9daa11c7b2848a2922` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:46.904056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ae29a-bc0b-4335-8f4c-3a2b89ce6856 None admin] RBAC: Authorizing `identity:create_grant(project_id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user_id=1863336fecad4987acdfd0b3f7b2f70f, role_id=6e93fc836fe74c9daa11c7b2848a2922)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:46.905757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6ae29a-bc0b-4335-8f4c-3a2b89ce6856 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:46.930714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 299/602] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 18:59:46 2022] PUT /identity/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/1863336fecad4987acdfd0b3f7b2f70f/roles/6e93fc836fe74c9daa11c7b2848a2922 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 18:59:46.930714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ae29a-bc0b-4335-8f4c-3a2b89ce6856 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:46.965945 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00497e4a-360a-4d2c-b673-6d46bf38a984 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:47.029653 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04de4c6-36a1-4747-b0a2-e9829cb28461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:47.030355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04de4c6-36a1-4747-b0a2-e9829cb28461 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:47.030637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04de4c6-36a1-4747-b0a2-e9829cb28461 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:47.030932 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04de4c6-36a1-4747-b0a2-e9829cb28461 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:47.034095 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:47.043751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04de4c6-36a1-4747-b0a2-e9829cb28461 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:47.044252 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:47.045455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04de4c6-36a1-4747-b0a2-e9829cb28461 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:47.061699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-e04de4c6-36a1-4747-b0a2-e9829cb28461 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:47.061699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:47.061699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:47.061699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:47.061699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:47.061699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:47.061699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:47.061699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:47.061699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:47.061699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:47.061699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:47.061699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:47.061699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:47.061699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:47.061699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:47.061699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:47.062920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:47.063914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:47.063914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:47.063914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:47.063914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 304/603] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:47.064436 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04de4c6-36a1-4747-b0a2-e9829cb28461 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:47.097942 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6ae29a-bc0b-4335-8f4c-3a2b89ce6856 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:47.157228 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4edf1a-a0ca-4b43-bf2c-2996b0a6c7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:47.158015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4edf1a-a0ca-4b43-bf2c-2996b0a6c7bb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:47.158404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4edf1a-a0ca-4b43-bf2c-2996b0a6c7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:47.158675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4edf1a-a0ca-4b43-bf2c-2996b0a6c7bb None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:47.159400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4edf1a-a0ca-4b43-bf2c-2996b0a6c7bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:47.159714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4edf1a-a0ca-4b43-bf2c-2996b0a6c7bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:47.161319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4edf1a-a0ca-4b43-bf2c-2996b0a6c7bb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:47.179924 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04de4c6-36a1-4747-b0a2-e9829cb28461 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:47.180106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 300/604] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 18:59:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:47.180913 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4edf1a-a0ca-4b43-bf2c-2996b0a6c7bb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:47.229510 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba18fb-20cb-42f9-bd0a-35f3a0e9a45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:47.231200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba18fb-20cb-42f9-bd0a-35f3a0e9a45f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:47.231711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba18fb-20cb-42f9-bd0a-35f3a0e9a45f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:47.232657 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba18fb-20cb-42f9-bd0a-35f3a0e9a45f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:47.248338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ba18fb-20cb-42f9-bd0a-35f3a0e9a45f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:47.256797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-81ba18fb-20cb-42f9-bd0a-35f3a0e9a45f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:47.256797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:47.256797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:47.256797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:47.256797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:47.256797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:47.256797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:47.256797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:47.256797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:47.256797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:47.256797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:47.256797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:47.256797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:47.256797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:47.256797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:47.256797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:47.257794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:47.257794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:47.257794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:47.257794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:47.257794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:47.257794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:47.257794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:47.257794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:47.257794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:47.257794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:47.257794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:47.257794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:47.257794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:47.257794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:47.257794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:47.257794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:47.258645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:47.258645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:47.258645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:47.258645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:47.258645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:47.258645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:47.258645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:47.258645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:47.258645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:47.258645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:47.258645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:47.258645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:47.261153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 305/605] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:47.262221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba18fb-20cb-42f9-bd0a-35f3a0e9a45f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:47.315995 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4edf1a-a0ca-4b43-bf2c-2996b0a6c7bb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:47.317914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:47.348804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:47.364607 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202ff5a-1796-42da-940a-76811b9e3c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:47.365269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202ff5a-1796-42da-940a-76811b9e3c85 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:47.365552 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202ff5a-1796-42da-940a-76811b9e3c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:47.365824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6202ff5a-1796-42da-940a-76811b9e3c85 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:47.366462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202ff5a-1796-42da-940a-76811b9e3c85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:47.366760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202ff5a-1796-42da-940a-76811b9e3c85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:47.368121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6202ff5a-1796-42da-940a-76811b9e3c85 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:47.390091 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba18fb-20cb-42f9-bd0a-35f3a0e9a45f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:47.390428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 301/606] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:47.391671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202ff5a-1796-42da-940a-76811b9e3c85 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:47.430459 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be388f-fdfa-471f-a71e-3fa5d61bba9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:47.431124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be388f-fdfa-471f-a71e-3fa5d61bba9a None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:47.431406 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be388f-fdfa-471f-a71e-3fa5d61bba9a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:47.431681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be388f-fdfa-471f-a71e-3fa5d61bba9a None admin] PATH_INFO: `/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/5d157abcf94745a5b92d6df64d872f73/roles/6e93fc836fe74c9daa11c7b2848a2922` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:47.450054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be388f-fdfa-471f-a71e-3fa5d61bba9a None admin] RBAC: Authorizing `identity:create_grant(project_id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user_id=5d157abcf94745a5b92d6df64d872f73, role_id=6e93fc836fe74c9daa11c7b2848a2922)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:47.451789 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be388f-fdfa-471f-a71e-3fa5d61bba9a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:47.481378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 306/607] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 18:59:47 2022] PUT /identity/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/5d157abcf94745a5b92d6df64d872f73/roles/6e93fc836fe74c9daa11c7b2848a2922 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 18:59:47.483167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be388f-fdfa-471f-a71e-3fa5d61bba9a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:47.526866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6202ff5a-1796-42da-940a-76811b9e3c85 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:47.570897 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0ee78e-3362-45f4-839a-9e6f6639e572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:47.571528 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ee78e-3362-45f4-839a-9e6f6639e572 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:47.571809 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ee78e-3362-45f4-839a-9e6f6639e572 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:47.572085 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0ee78e-3362-45f4-839a-9e6f6639e572 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:47.573142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ee78e-3362-45f4-839a-9e6f6639e572 None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:47.573484 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ee78e-3362-45f4-839a-9e6f6639e572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:47.575496 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0ee78e-3362-45f4-839a-9e6f6639e572 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:47.591739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 302/608] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 18:59:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:47.591739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10362988-412a-4e3a-8efe-52576fceec6d None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:47.591739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10362988-412a-4e3a-8efe-52576fceec6d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:47.591739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10362988-412a-4e3a-8efe-52576fceec6d None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:47.591739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 303/609] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:47.596582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10362988-412a-4e3a-8efe-52576fceec6d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:47.637456 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:47.642653 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be388f-fdfa-471f-a71e-3fa5d61bba9a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:47.660047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:47.697058 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c2d13-270f-4668-897c-a76c7d422adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:47.698169 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c2d13-270f-4668-897c-a76c7d422adb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:47.698667 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c2d13-270f-4668-897c-a76c7d422adb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:47.699170 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c2d13-270f-4668-897c-a76c7d422adb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:47.710703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c2d13-270f-4668-897c-a76c7d422adb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:47.713424 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c2d13-270f-4668-897c-a76c7d422adb None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:47.726159 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10362988-412a-4e3a-8efe-52576fceec6d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:47.731585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-ce2c2d13-270f-4668-897c-a76c7d422adb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:47.731585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:47.731585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:47.731585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:47.731585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:47.731585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:47.731585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:47.731585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:47.731585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:47.731585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:47.731585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:47.731585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:47.731585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 18:59:47.731585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 18:59:47.731585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:47.732505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:47.733440 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 18:59:47.733440 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 18:59:47.733440 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 18:59:47.733440 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 18:59:47.733440 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 18:59:47.733440 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 18:59:47.733440 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:47.733440 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:47.736154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 307/610] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:47.738531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349a6e6-c618-4a8c-b0f6-155dff765d54 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:47.739185 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349a6e6-c618-4a8c-b0f6-155dff765d54 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:47.739766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349a6e6-c618-4a8c-b0f6-155dff765d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:47.763638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a349a6e6-c618-4a8c-b0f6-155dff765d54 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:47.764966 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a349a6e6-c618-4a8c-b0f6-155dff765d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:47.772526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6887861-375f-47ae-9d6a-881696147bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:47.774012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6887861-375f-47ae-9d6a-881696147bdd None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:47.774545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6887861-375f-47ae-9d6a-881696147bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:47.775543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6887861-375f-47ae-9d6a-881696147bdd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:47.790426 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6887861-375f-47ae-9d6a-881696147bdd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:47.793450 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6887861-375f-47ae-9d6a-881696147bdd None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:47.813216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-d6887861-375f-47ae-9d6a-881696147bdd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:47.813216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:47.813216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:47.813216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:47.813216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:47.813216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:47.813216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:47.813216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:47.813216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:47.813216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:47.813216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:47.813216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:47.813216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:47.813216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:47.813216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:47.813216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:47.814223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:47.815146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:47.815146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:47.815146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:47.822586 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 304/611] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:47.822586 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6887861-375f-47ae-9d6a-881696147bdd None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:47.847580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 308/612] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:47.848943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349a6e6-c618-4a8c-b0f6-155dff765d54 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:47.924474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:47.928116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6887861-375f-47ae-9d6a-881696147bdd None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:47.949796 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:47.964439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349a6e6-c618-4a8c-b0f6-155dff765d54 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:47.988132 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2be2e-201a-4d5a-b5e4-35d066f60d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:47.988454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2be2e-201a-4d5a-b5e4-35d066f60d16 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:47.988717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2be2e-201a-4d5a-b5e4-35d066f60d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:47.988983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de2be2e-201a-4d5a-b5e4-35d066f60d16 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:47.989697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2be2e-201a-4d5a-b5e4-35d066f60d16 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:47.989995 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2be2e-201a-4d5a-b5e4-35d066f60d16 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:47.991402 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de2be2e-201a-4d5a-b5e4-35d066f60d16 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:48.004420 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 305/613] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 18:59:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:48.004788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe0676-8b01-4f9e-bb0e-4e4b8a508388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:48.005641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe0676-8b01-4f9e-bb0e-4e4b8a508388 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:48.006127 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe0676-8b01-4f9e-bb0e-4e4b8a508388 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:48.006596 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe0676-8b01-4f9e-bb0e-4e4b8a508388 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:48.007509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe0676-8b01-4f9e-bb0e-4e4b8a508388 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:48.008007 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe0676-8b01-4f9e-bb0e-4e4b8a508388 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:48.009761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbe0676-8b01-4f9e-bb0e-4e4b8a508388 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:48.010045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2be2e-201a-4d5a-b5e4-35d066f60d16 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:48.024277 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 309/614] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:48.030778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe0676-8b01-4f9e-bb0e-4e4b8a508388 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:48.201112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de2be2e-201a-4d5a-b5e4-35d066f60d16 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:48.205746 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe0676-8b01-4f9e-bb0e-4e4b8a508388 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:48.246722 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53653b83-a04e-4b11-aa9c-c0d61ab31e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:48.247299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53653b83-a04e-4b11-aa9c-c0d61ab31e53 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:48.247576 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53653b83-a04e-4b11-aa9c-c0d61ab31e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:48.247845 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53653b83-a04e-4b11-aa9c-c0d61ab31e53 None admin] PATH_INFO: `/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/093edc8c192c469f9532a45bb3326612/roles/6e93fc836fe74c9daa11c7b2848a2922` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:48.255704 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2762a2-b7b7-4666-af00-625f590f47b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:48.256548 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2762a2-b7b7-4666-af00-625f590f47b4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:48.256992 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2762a2-b7b7-4666-af00-625f590f47b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:48.257428 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2762a2-b7b7-4666-af00-625f590f47b4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:48.263548 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53653b83-a04e-4b11-aa9c-c0d61ab31e53 None admin] RBAC: Authorizing `identity:create_grant(project_id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user_id=093edc8c192c469f9532a45bb3326612, role_id=6e93fc836fe74c9daa11c7b2848a2922)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:48.264991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53653b83-a04e-4b11-aa9c-c0d61ab31e53 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:48.266381 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2762a2-b7b7-4666-af00-625f590f47b4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:48.267964 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2762a2-b7b7-4666-af00-625f590f47b4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:48.281335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-5b2762a2-b7b7-4666-af00-625f590f47b4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 18:59:48.281335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:48.281335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:48.281335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:48.281335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:48.281335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:48.281335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:48.281335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:48.281335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:48.281335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:48.281335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:48.281335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:48.281335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 18:59:48.281335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 18:59:48.281335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 18:59:48.281335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:48.282433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:48.283675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:48.283675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:48.283675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 18:59:48.283675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 18:59:48.283675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 18:59:48.283675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 18:59:48.283675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 18:59:48.283675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 18:59:48.283675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 18:59:48.283675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:48.284208 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 306/615] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 18:59:48 2022] PUT /identity/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/093edc8c192c469f9532a45bb3326612/roles/6e93fc836fe74c9daa11c7b2848a2922 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 18:59:48.284208 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53653b83-a04e-4b11-aa9c-c0d61ab31e53 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:48.285923 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 310/616] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 18:59:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:48.297730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:48.299362 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2762a2-b7b7-4666-af00-625f590f47b4 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:48.330203 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:48.440689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53653b83-a04e-4b11-aa9c-c0d61ab31e53 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:48.492961 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8316e-fbda-4ef0-a7c3-5a4c25422419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:48.493740 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8316e-fbda-4ef0-a7c3-5a4c25422419 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:48.494026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8316e-fbda-4ef0-a7c3-5a4c25422419 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:48.494288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8316e-fbda-4ef0-a7c3-5a4c25422419 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:48.507152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc8316e-fbda-4ef0-a7c3-5a4c25422419 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:48.510374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2762a2-b7b7-4666-af00-625f590f47b4 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:48.513143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-9dc8316e-fbda-4ef0-a7c3-5a4c25422419 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:48.513143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:48.513143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:48.513143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:48.513143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:48.513143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:48.513143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:48.513143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:48.513143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:48.513143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:48.513143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:48.513143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:48.513143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:48.513143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:48.513143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:48.513143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:48.514061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:48.514061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:48.514061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:48.514061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:48.514061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:48.514061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:48.514061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:48.514061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:48.514061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:48.514061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:48.514061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:48.514061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:48.514061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:48.514061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:48.514061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:48.514061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:48.515230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:48.515230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:48.515230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:48.515230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:48.515230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:48.515230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:48.515230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:48.515230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:48.515230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:48.515230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:48.515230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:48.515230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:48.515230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 307/617] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:48.527935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8316e-fbda-4ef0-a7c3-5a4c25422419 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:48.556833 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805922e1-d0f2-4383-b4f2-da02ab78d034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:48.557473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805922e1-d0f2-4383-b4f2-da02ab78d034 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:48.557918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805922e1-d0f2-4383-b4f2-da02ab78d034 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:48.558198 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805922e1-d0f2-4383-b4f2-da02ab78d034 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:48.559298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805922e1-d0f2-4383-b4f2-da02ab78d034 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:48.559875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805922e1-d0f2-4383-b4f2-da02ab78d034 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:48.561499 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805922e1-d0f2-4383-b4f2-da02ab78d034 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:48.584184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 311/618] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 18:59:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:48.590599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805922e1-d0f2-4383-b4f2-da02ab78d034 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:48.629554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:48.689573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:48.693326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8316e-fbda-4ef0-a7c3-5a4c25422419 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:48.748162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805922e1-d0f2-4383-b4f2-da02ab78d034 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:48.772886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d452b-a51d-412d-8d36-63bf1a49966c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:48.773486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d452b-a51d-412d-8d36-63bf1a49966c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:48.773764 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d452b-a51d-412d-8d36-63bf1a49966c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:48.774033 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d452b-a51d-412d-8d36-63bf1a49966c None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:48.774695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d452b-a51d-412d-8d36-63bf1a49966c None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:48.774994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d452b-a51d-412d-8d36-63bf1a49966c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:48.777046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187d452b-a51d-412d-8d36-63bf1a49966c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:48.788843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 308/619] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 18:59:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:48.794796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d452b-a51d-412d-8d36-63bf1a49966c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:48.837901 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f00ff-a19f-4685-89fd-be3d4ab6d84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:48.838528 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f00ff-a19f-4685-89fd-be3d4ab6d84e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:48.838810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f00ff-a19f-4685-89fd-be3d4ab6d84e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:48.839084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769f00ff-a19f-4685-89fd-be3d4ab6d84e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:48.936588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d452b-a51d-412d-8d36-63bf1a49966c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:48.960952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f00ff-a19f-4685-89fd-be3d4ab6d84e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:48.962424 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769f00ff-a19f-4685-89fd-be3d4ab6d84e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:48.993557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:49.001311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050ce00-3e44-4922-851d-86720f155171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:49.001941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050ce00-3e44-4922-851d-86720f155171 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:49.002219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050ce00-3e44-4922-851d-86720f155171 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:49.002483 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d050ce00-3e44-4922-851d-86720f155171 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:49.022458 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050ce00-3e44-4922-851d-86720f155171 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:49.023964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d050ce00-3e44-4922-851d-86720f155171 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:49.051559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-d050ce00-3e44-4922-851d-86720f155171 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:49.051559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:49.051559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:49.051559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:49.051559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:49.051559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:49.051559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:49.051559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:49.051559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:49.051559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:49.051559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:49.051559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:49.051559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:49.051559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:49.051559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:49.051559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:49.052551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:49.053468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:49.053468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:49.053468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:49.054048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 309/620] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:49.062110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050ce00-3e44-4922-851d-86720f155171 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:49.078023 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:49.149495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 312/621] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:49.153364 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f00ff-a19f-4685-89fd-be3d4ab6d84e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:49.223402 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d050ce00-3e44-4922-851d-86720f155171 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:49.267440 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c9e3ae-d99d-4a83-bab9-6454059d9f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:49.268692 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9e3ae-d99d-4a83-bab9-6454059d9f72 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:49.269514 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9e3ae-d99d-4a83-bab9-6454059d9f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:49.269987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c9e3ae-d99d-4a83-bab9-6454059d9f72 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:49.271264 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9e3ae-d99d-4a83-bab9-6454059d9f72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:49.271817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9e3ae-d99d-4a83-bab9-6454059d9f72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:49.273641 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c9e3ae-d99d-4a83-bab9-6454059d9f72 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:49.280633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769f00ff-a19f-4685-89fd-be3d4ab6d84e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:49.290131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 310/622] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:49.292064 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b28fc-de51-483f-80e0-2ebe8b2f8673 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:49.292504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b28fc-de51-483f-80e0-2ebe8b2f8673 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:49.293608 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b28fc-de51-483f-80e0-2ebe8b2f8673 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:49.299527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 311/623] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:49.299527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b28fc-de51-483f-80e0-2ebe8b2f8673 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:49.339815 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90db5565-a6b4-41fa-b5bd-cdc06c0dcaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:49.340799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db5565-a6b4-41fa-b5bd-cdc06c0dcaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:49.341486 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db5565-a6b4-41fa-b5bd-cdc06c0dcaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:49.342801 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90db5565-a6b4-41fa-b5bd-cdc06c0dcaa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:49.351668 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:49.357886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db5565-a6b4-41fa-b5bd-cdc06c0dcaa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:49.360056 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90db5565-a6b4-41fa-b5bd-cdc06c0dcaa7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:49.381173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-90db5565-a6b4-41fa-b5bd-cdc06c0dcaa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:49.381173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:49.381173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:49.381173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:49.381173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:49.381173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:49.381173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:49.381173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:49.381173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:49.381173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:49.381173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:49.381173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:49.381173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:49.381173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:49.381173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:49.381173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:49.382201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:49.383092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:49.383092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:49.383092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:49.385930 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 313/624] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:49.387314 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef91f4-6704-4b8b-a81e-9aedeea62c3c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:49.387731 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef91f4-6704-4b8b-a81e-9aedeea62c3c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:49.388163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef91f4-6704-4b8b-a81e-9aedeea62c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:49.411841 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-baef91f4-6704-4b8b-a81e-9aedeea62c3c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:49.413796 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-baef91f4-6704-4b8b-a81e-9aedeea62c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:49.418879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b28fc-de51-483f-80e0-2ebe8b2f8673 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:49.432321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:49.457743 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9643ba3-d6eb-4de6-904f-2ad38b4ffe64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:49.458518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9643ba3-d6eb-4de6-904f-2ad38b4ffe64 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:49.458976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9643ba3-d6eb-4de6-904f-2ad38b4ffe64 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:49.459351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9643ba3-d6eb-4de6-904f-2ad38b4ffe64 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:49.468831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9643ba3-d6eb-4de6-904f-2ad38b4ffe64 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:49.471356 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9643ba3-d6eb-4de6-904f-2ad38b4ffe64 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:49.488389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-a9643ba3-d6eb-4de6-904f-2ad38b4ffe64 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 18:59:49.488389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:49.488389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:49.488389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:49.488389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:49.488389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:49.488389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:49.488389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:49.488389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:49.488389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:49.488389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:49.488389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:49.488389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 18:59:49.488389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 18:59:49.488389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 18:59:49.488389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:49.489551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:49.490502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:49.490502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:49.490502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 18:59:49.490502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 18:59:49.490502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 18:59:49.490502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 18:59:49.490502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 18:59:49.490502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 18:59:49.490502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 18:59:49.490502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:49.493253 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 312/625] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:59:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:49.494299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9643ba3-d6eb-4de6-904f-2ad38b4ffe64 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:49.513260 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 314/626] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:49.513260 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef91f4-6704-4b8b-a81e-9aedeea62c3c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:49.616673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef91f4-6704-4b8b-a81e-9aedeea62c3c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:49.653117 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7458e8-0b35-428d-b69d-fd39e839f68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:49.653715 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7458e8-0b35-428d-b69d-fd39e839f68d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:49.654044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7458e8-0b35-428d-b69d-fd39e839f68d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:49.654319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7458e8-0b35-428d-b69d-fd39e839f68d None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:49.655154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7458e8-0b35-428d-b69d-fd39e839f68d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:49.655463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7458e8-0b35-428d-b69d-fd39e839f68d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:49.656894 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7458e8-0b35-428d-b69d-fd39e839f68d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:49.669398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9643ba3-d6eb-4de6-904f-2ad38b4ffe64 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:49.671648 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:49.677475 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 315/627] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 31 18:59:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:49.689227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7458e8-0b35-428d-b69d-fd39e839f68d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:49.692615 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:49.715127 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d217925e-9b90-4a4c-9a55-9543c157879c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:49.715781 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217925e-9b90-4a4c-9a55-9543c157879c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:49.716061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217925e-9b90-4a4c-9a55-9543c157879c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:49.716328 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d217925e-9b90-4a4c-9a55-9543c157879c None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:49.716965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217925e-9b90-4a4c-9a55-9543c157879c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:49.717301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217925e-9b90-4a4c-9a55-9543c157879c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:49.718732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d217925e-9b90-4a4c-9a55-9543c157879c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:49.736304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 313/628] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:49.737669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a24281-ceed-4450-b06b-3cc1c5cb2230 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:49.737954 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a24281-ceed-4450-b06b-3cc1c5cb2230 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:49.738229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a24281-ceed-4450-b06b-3cc1c5cb2230 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:49.739743 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 314/629] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:49.740442 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a24281-ceed-4450-b06b-3cc1c5cb2230 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:49.794720 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7458e8-0b35-428d-b69d-fd39e839f68d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:49.832816 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6755827f-9573-45ab-9df0-020d50079119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:49.833471 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755827f-9573-45ab-9df0-020d50079119 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:49.833751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755827f-9573-45ab-9df0-020d50079119 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:49.834023 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6755827f-9573-45ab-9df0-020d50079119 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:49.845951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755827f-9573-45ab-9df0-020d50079119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:49.847386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6755827f-9573-45ab-9df0-020d50079119 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:49.849314 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a24281-ceed-4450-b06b-3cc1c5cb2230 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:49.863937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-6755827f-9573-45ab-9df0-020d50079119 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:49.863937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:49.863937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:49.863937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:49.863937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:49.863937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:49.863937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:49.863937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:49.863937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:49.863937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:49.863937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:49.863937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:49.863937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:49.863937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:49.863937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:49.863937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:49.865074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:49.866014 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:49.866014 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:49.866014 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:49.870244 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 316/630] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 18:59:49.871854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd13472-089b-4252-9624-30b56e7d4734 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:49.872137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd13472-089b-4252-9624-30b56e7d4734 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:49.872408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd13472-089b-4252-9624-30b56e7d4734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:49.896800 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4fd13472-089b-4252-9624-30b56e7d4734 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:49.898169 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4fd13472-089b-4252-9624-30b56e7d4734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:49.903513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c937e0-bd82-4750-8d8c-739d91b32fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:49.904155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c937e0-bd82-4750-8d8c-739d91b32fa2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:49.904422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c937e0-bd82-4750-8d8c-739d91b32fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:49.904690 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c937e0-bd82-4750-8d8c-739d91b32fa2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:49.913346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c937e0-bd82-4750-8d8c-739d91b32fa2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:49.915294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c937e0-bd82-4750-8d8c-739d91b32fa2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:49.929972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-04c937e0-bd82-4750-8d8c-739d91b32fa2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:49.929972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:49.929972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:49.929972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:49.929972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:49.929972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:49.929972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:49.929972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:49.929972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:49.929972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:49.929972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:49.929972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:49.929972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 18:59:49.929972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 18:59:49.929972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:49.930918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:49.931860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 18:59:49.931860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 18:59:49.931860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 18:59:49.931860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 18:59:49.931860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 18:59:49.931860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 18:59:49.931860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:49.931860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:49.931860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 315/631] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:49.932356 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c937e0-bd82-4750-8d8c-739d91b32fa2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:49.959241 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:49.974156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:50.001229 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 317/632] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:50.001229 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd13472-089b-4252-9624-30b56e7d4734 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:50.047319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c937e0-bd82-4750-8d8c-739d91b32fa2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:50.093079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa965b5-2ce3-41e9-9228-5c165488ac13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:50.093713 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa965b5-2ce3-41e9-9228-5c165488ac13 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:50.093985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa965b5-2ce3-41e9-9228-5c165488ac13 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:50.094251 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa965b5-2ce3-41e9-9228-5c165488ac13 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:50.094885 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa965b5-2ce3-41e9-9228-5c165488ac13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:50.095181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa965b5-2ce3-41e9-9228-5c165488ac13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:50.097255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa965b5-2ce3-41e9-9228-5c165488ac13 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:50.107880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd13472-089b-4252-9624-30b56e7d4734 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:50.112763 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 316/633] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:50.113612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa965b5-2ce3-41e9-9228-5c165488ac13 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:50.147086 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44188c-bb39-4d23-a86f-7ef141093e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:50.147859 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44188c-bb39-4d23-a86f-7ef141093e4c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:50.148233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44188c-bb39-4d23-a86f-7ef141093e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:50.148598 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44188c-bb39-4d23-a86f-7ef141093e4c None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:50.149417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44188c-bb39-4d23-a86f-7ef141093e4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:50.149904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44188c-bb39-4d23-a86f-7ef141093e4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:50.151519 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e44188c-bb39-4d23-a86f-7ef141093e4c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:50.164690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 318/634] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 18:59:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:50.165914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44188c-bb39-4d23-a86f-7ef141093e4c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:50.217905 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:50.228624 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:50.239504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa965b5-2ce3-41e9-9228-5c165488ac13 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:50.275413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1455f-304d-46dd-aa48-0f14f16e4f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:50.276042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1455f-304d-46dd-aa48-0f14f16e4f7d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:50.276308 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1455f-304d-46dd-aa48-0f14f16e4f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:50.276575 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b1455f-304d-46dd-aa48-0f14f16e4f7d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:50.287294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b1455f-304d-46dd-aa48-0f14f16e4f7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:50.287419 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e44188c-bb39-4d23-a86f-7ef141093e4c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:50.292008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-11b1455f-304d-46dd-aa48-0f14f16e4f7d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:50.292008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:50.292008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:50.292008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:50.292008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:50.292008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:50.292008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:50.292008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:50.292008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:50.292008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:50.292008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:50.292008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:50.292008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:50.292008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:50.292008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:50.292008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:50.293003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:50.293003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:50.293003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:50.293003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:50.293003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:50.293003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:50.293003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:50.293003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:50.293003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:50.293003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:50.293003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:50.293003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:50.293003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:50.293003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:50.293003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:50.293003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:50.294192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:50.294192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:50.294192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:50.294192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:50.294192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:50.294192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:50.294192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:50.294192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:50.294192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:50.294192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:50.294192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:50.294192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:50.297860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 317/635] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:50.298612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1455f-304d-46dd-aa48-0f14f16e4f7d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:50.326580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3787041-668d-4e99-bb36-b04ff7396bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:50.327203 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3787041-668d-4e99-bb36-b04ff7396bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:50.327483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3787041-668d-4e99-bb36-b04ff7396bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:50.327750 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3787041-668d-4e99-bb36-b04ff7396bf6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:50.334909 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3787041-668d-4e99-bb36-b04ff7396bf6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:50.336335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3787041-668d-4e99-bb36-b04ff7396bf6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:50.348681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-e3787041-668d-4e99-bb36-b04ff7396bf6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:50.348681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:50.348681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:50.348681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:50.348681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:50.348681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:50.348681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:50.348681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:50.348681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:50.348681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:50.348681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:50.348681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:50.348681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 18:59:50.348681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 18:59:50.348681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:50.349629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:50.350524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 18:59:50.350524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 18:59:50.350524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 18:59:50.350524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 18:59:50.350524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 18:59:50.350524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 18:59:50.350524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:50.350524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:50.350524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 319/636] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:50.350524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3787041-668d-4e99-bb36-b04ff7396bf6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:50.409098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b1455f-304d-46dd-aa48-0f14f16e4f7d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:50.443940 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a23179-5313-4268-a459-19cd1be48a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:50.444537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a23179-5313-4268-a459-19cd1be48a8d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:50.444813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a23179-5313-4268-a459-19cd1be48a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:50.445082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a23179-5313-4268-a459-19cd1be48a8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:50.445821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a23179-5313-4268-a459-19cd1be48a8d None admin] RBAC: Adding query filter params (role.id=6e93fc836fe74c9daa11c7b2848a2922, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=673a72ac9a9847e3812007b66ac7a5ea) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:50.446122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a23179-5313-4268-a459-19cd1be48a8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:50.447537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a23179-5313-4268-a459-19cd1be48a8d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:50.453870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 318/637] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:59:50 2022] GET /identity/v3/role_assignments?user.id=673a72ac9a9847e3812007b66ac7a5ea&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=6e93fc836fe74c9daa11c7b2848a2922 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:50.454639 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3787041-668d-4e99-bb36-b04ff7396bf6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:50.454804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a23179-5313-4268-a459-19cd1be48a8d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:50.464413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:50.472208 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:50.494110 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa2f99-593e-4039-8fb1-8c956b31d87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:50.494740 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa2f99-593e-4039-8fb1-8c956b31d87f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:50.495016 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa2f99-593e-4039-8fb1-8c956b31d87f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:50.495318 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa2f99-593e-4039-8fb1-8c956b31d87f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:50.507852 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa2f99-593e-4039-8fb1-8c956b31d87f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:50.513937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-92fa2f99-593e-4039-8fb1-8c956b31d87f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:50.513937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:50.513937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:50.513937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:50.513937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:50.513937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:50.513937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:50.513937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:50.513937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:50.513937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:50.513937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:50.513937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:50.513937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:50.513937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:50.513937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:50.513937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:50.514937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:50.514937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:50.514937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:50.514937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:50.514937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:50.514937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:50.514937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:50.514937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:50.514937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:50.514937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:50.514937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:50.514937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:50.514937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:50.514937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:50.514937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:50.514937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:50.515768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:50.515768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:50.515768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:50.515768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:50.515768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:50.515768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:50.515768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:50.515768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:50.515768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:50.515768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:50.515768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:50.515768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:50.515768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 320/638] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:50.516444 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa2f99-593e-4039-8fb1-8c956b31d87f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:50.585036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a23179-5313-4268-a459-19cd1be48a8d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:50.626030 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa2f99-593e-4039-8fb1-8c956b31d87f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:50.626422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dc879b-f640-416c-ae47-53948afdceab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:50.627037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc879b-f640-416c-ae47-53948afdceab None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:50.627317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc879b-f640-416c-ae47-53948afdceab None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:50.627590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dc879b-f640-416c-ae47-53948afdceab None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:50.628256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc879b-f640-416c-ae47-53948afdceab None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:50.628561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc879b-f640-416c-ae47-53948afdceab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:50.629975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dc879b-f640-416c-ae47-53948afdceab None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:50.638907 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 319/639] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 18:59:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:50.640925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dac3f7-dd14-484a-81a4-06d26264fb56 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:50.641233 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dac3f7-dd14-484a-81a4-06d26264fb56 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:50.641511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dac3f7-dd14-484a-81a4-06d26264fb56 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:50.642786 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 320/640] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:50.643542 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dac3f7-dd14-484a-81a4-06d26264fb56 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:50.674188 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da54c2-1472-49ea-93d0-2c86a0fa817c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:50.674730 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da54c2-1472-49ea-93d0-2c86a0fa817c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:50.675009 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da54c2-1472-49ea-93d0-2c86a0fa817c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:50.675280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0da54c2-1472-49ea-93d0-2c86a0fa817c None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:50.675980 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da54c2-1472-49ea-93d0-2c86a0fa817c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:50.676284 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da54c2-1472-49ea-93d0-2c86a0fa817c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:50.678002 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0da54c2-1472-49ea-93d0-2c86a0fa817c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:50.691430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 321/641] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 18:59:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:50.692248 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da54c2-1472-49ea-93d0-2c86a0fa817c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:50.722006 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:50.732198 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:50.768297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dac3f7-dd14-484a-81a4-06d26264fb56 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:50.799517 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0da54c2-1472-49ea-93d0-2c86a0fa817c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:50.802912 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fb9c4-734f-42fc-a0cc-28e8b396518c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:50.803520 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fb9c4-734f-42fc-a0cc-28e8b396518c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:50.803799 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fb9c4-734f-42fc-a0cc-28e8b396518c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:50.804071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fb9c4-734f-42fc-a0cc-28e8b396518c None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:50.804729 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fb9c4-734f-42fc-a0cc-28e8b396518c None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:50.805030 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fb9c4-734f-42fc-a0cc-28e8b396518c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:50.806420 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fb9c4-734f-42fc-a0cc-28e8b396518c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:50.815845 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 321/642] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 18:59:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:50.817022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788ecd7-ec5f-4da9-9401-3fab067f7d2d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:50.817335 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788ecd7-ec5f-4da9-9401-3fab067f7d2d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:50.817611 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2788ecd7-ec5f-4da9-9401-3fab067f7d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:50.836007 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2788ecd7-ec5f-4da9-9401-3fab067f7d2d None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:50.837459 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2788ecd7-ec5f-4da9-9401-3fab067f7d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:50.841313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e94af-873a-4614-ae3c-5f16b0c05a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:50.841941 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e94af-873a-4614-ae3c-5f16b0c05a8b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:50.842223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e94af-873a-4614-ae3c-5f16b0c05a8b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:50.842504 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e94af-873a-4614-ae3c-5f16b0c05a8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:50.852891 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e94af-873a-4614-ae3c-5f16b0c05a8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:50.854364 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e94af-873a-4614-ae3c-5f16b0c05a8b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:50.870012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-a29e94af-873a-4614-ae3c-5f16b0c05a8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:50.870012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:50.870012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:50.870012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:50.870012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:50.870012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:50.870012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:50.870012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:50.870012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:50.870012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:50.870012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:50.870012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:50.870012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:50.870012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:50.870012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:50.870012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:50.871109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:50.872513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:50.872513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:50.872513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:50.872513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 322/643] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:50.872513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e94af-873a-4614-ae3c-5f16b0c05a8b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:50.929301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 322/644] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:50.929301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788ecd7-ec5f-4da9-9401-3fab067f7d2d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:50.984571 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:50.984913 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:51.011306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e94af-873a-4614-ae3c-5f16b0c05a8b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:51.029719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2788ecd7-ec5f-4da9-9401-3fab067f7d2d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:51.058280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419243da-f9f6-47ef-900a-1630ea3485e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:51.059457 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419243da-f9f6-47ef-900a-1630ea3485e7 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:51.059887 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419243da-f9f6-47ef-900a-1630ea3485e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:51.060380 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419243da-f9f6-47ef-900a-1630ea3485e7 None admin] PATH_INFO: `/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/586715d11b924e2cb6f44030efd77619/roles/6e93fc836fe74c9daa11c7b2848a2922` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:51.066284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7c04d-d5fc-496b-9428-b761a6dcc6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:51.066912 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7c04d-d5fc-496b-9428-b761a6dcc6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:51.067191 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7c04d-d5fc-496b-9428-b761a6dcc6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:51.067463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e7c04d-d5fc-496b-9428-b761a6dcc6e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:51.080782 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419243da-f9f6-47ef-900a-1630ea3485e7 None admin] RBAC: Authorizing `identity:create_grant(project_id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user_id=586715d11b924e2cb6f44030efd77619, role_id=6e93fc836fe74c9daa11c7b2848a2922)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:51.082338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e7c04d-d5fc-496b-9428-b761a6dcc6e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:51.082735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419243da-f9f6-47ef-900a-1630ea3485e7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:51.083812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e7c04d-d5fc-496b-9428-b761a6dcc6e2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:51.099187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 323/645] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 18:59:50 2022] PUT /identity/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/586715d11b924e2cb6f44030efd77619/roles/6e93fc836fe74c9daa11c7b2848a2922 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 18:59:51.099543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-70e7c04d-d5fc-496b-9428-b761a6dcc6e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:51.099543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:51.099543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:51.099543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:51.099543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:51.099543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:51.099543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:51.099543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:51.099543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:51.099543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:51.099543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:51.099543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:51.099543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:51.099543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:51.099543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:51.099543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:51.100674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:51.101704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:51.101704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:51.101704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:51.101704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 323/646] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:51.102360 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419243da-f9f6-47ef-900a-1630ea3485e7 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:51.115220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7c04d-d5fc-496b-9428-b761a6dcc6e2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:51.272113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e7c04d-d5fc-496b-9428-b761a6dcc6e2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:51.288744 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:51.313430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419243da-f9f6-47ef-900a-1630ea3485e7 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:51.313901 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899cf335-9d4a-4240-a1a8-bd5ea8724d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:51.314511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cf335-9d4a-4240-a1a8-bd5ea8724d1b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:51.314787 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cf335-9d4a-4240-a1a8-bd5ea8724d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:51.315055 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899cf335-9d4a-4240-a1a8-bd5ea8724d1b None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:51.315699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899cf335-9d4a-4240-a1a8-bd5ea8724d1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:51.315996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899cf335-9d4a-4240-a1a8-bd5ea8724d1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:51.318173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899cf335-9d4a-4240-a1a8-bd5ea8724d1b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:51.331481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 324/647] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 18:59:51.332319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899cf335-9d4a-4240-a1a8-bd5ea8724d1b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:51.355257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:51.363893 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cea3af-d90c-4b25-9f6d-4bef8359adcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:51.365207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cea3af-d90c-4b25-9f6d-4bef8359adcf None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:51.365823 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cea3af-d90c-4b25-9f6d-4bef8359adcf None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:51.366467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cea3af-d90c-4b25-9f6d-4bef8359adcf None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:51.367526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cea3af-d90c-4b25-9f6d-4bef8359adcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:51.368416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cea3af-d90c-4b25-9f6d-4bef8359adcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:51.371048 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cea3af-d90c-4b25-9f6d-4bef8359adcf None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:51.387817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 324/648] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:51.388922 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cea3af-d90c-4b25-9f6d-4bef8359adcf None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:51.454831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899cf335-9d4a-4240-a1a8-bd5ea8724d1b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:51.492051 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03615c40-f34c-45b3-ae84-138c6e3319fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:51.492735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03615c40-f34c-45b3-ae84-138c6e3319fc None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:51.493075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03615c40-f34c-45b3-ae84-138c6e3319fc None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:51.493423 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03615c40-f34c-45b3-ae84-138c6e3319fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:51.506805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03615c40-f34c-45b3-ae84-138c6e3319fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:51.514196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-03615c40-f34c-45b3-ae84-138c6e3319fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:51.514196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:51.514196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:51.514196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:51.514196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:51.514196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:51.514196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:51.514196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:51.514196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:51.514196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:51.514196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:51.514196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:51.514196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:51.514196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:51.514196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:51.514196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:51.515131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:51.515131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:51.515131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:51.515131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:51.515131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:51.515131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:51.515131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:51.515131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:51.515131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:51.515131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:51.515131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:51.515131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:51.515131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:51.515131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:51.515131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:51.515131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:51.515969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:51.515969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:51.515969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:51.515969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:51.515969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:51.515969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:51.515969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:51.515969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:51.515969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:51.515969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:51.515969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:51.515969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:51.520528 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 325/649] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:51.521578 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03615c40-f34c-45b3-ae84-138c6e3319fc None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:51.528830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cea3af-d90c-4b25-9f6d-4bef8359adcf None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:51.562687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:51.577088 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969c083-8d9e-4088-9b33-1f51b0e752b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:51.578463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969c083-8d9e-4088-9b33-1f51b0e752b8 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:51.578887 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969c083-8d9e-4088-9b33-1f51b0e752b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:51.579289 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969c083-8d9e-4088-9b33-1f51b0e752b8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:51.590915 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969c083-8d9e-4088-9b33-1f51b0e752b8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:51.592565 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6969c083-8d9e-4088-9b33-1f51b0e752b8 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:51.608822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-6969c083-8d9e-4088-9b33-1f51b0e752b8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 18:59:51.608822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:51.608822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:51.608822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:51.608822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:51.608822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:51.608822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:51.608822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:51.608822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:51.608822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:51.608822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:51.608822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:51.608822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 18:59:51.608822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 18:59:51.608822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 18:59:51.608822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:51.609993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:51.611115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:51.611115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:51.611115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 18:59:51.611115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 18:59:51.611115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 18:59:51.611115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 18:59:51.611115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 18:59:51.611115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 18:59:51.611115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 18:59:51.611115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:51.613545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 325/650] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:51.614543 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969c083-8d9e-4088-9b33-1f51b0e752b8 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:51.653040 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03615c40-f34c-45b3-ae84-138c6e3319fc None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:51.672008 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:51.689877 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a08998-8b4f-4195-919f-d364d8326b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:51.690731 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a08998-8b4f-4195-919f-d364d8326b07 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:51.691188 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a08998-8b4f-4195-919f-d364d8326b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:51.691596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a08998-8b4f-4195-919f-d364d8326b07 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:51.699510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a08998-8b4f-4195-919f-d364d8326b07 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:51.701111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a08998-8b4f-4195-919f-d364d8326b07 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:51.713133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-14a08998-8b4f-4195-919f-d364d8326b07 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 31 18:59:51.713133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:51.713133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:51.713133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:51.713133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:51.713133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:51.713133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:51.713133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:51.713133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:51.713133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:51.713133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:51.713133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:51.713133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 18:59:51.713133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 18:59:51.713133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 18:59:51.713133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:51.714089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:51.715745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:51.715745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:51.715745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 18:59:51.715745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 18:59:51.715745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 18:59:51.715745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 18:59:51.715745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 18:59:51.715745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 18:59:51.715745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 31 18:59:51.715745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:51.718288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 326/651] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:51 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:51.719315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a08998-8b4f-4195-919f-d364d8326b07 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:51.734979 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969c083-8d9e-4088-9b33-1f51b0e752b8 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:51.771654 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7a63a-7bdd-41cc-b180-8a7ca9db752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:51.772346 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7a63a-7bdd-41cc-b180-8a7ca9db752e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:51.772717 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7a63a-7bdd-41cc-b180-8a7ca9db752e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:51.773027 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7a63a-7bdd-41cc-b180-8a7ca9db752e None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:51.773782 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7a63a-7bdd-41cc-b180-8a7ca9db752e None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:51.774380 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7a63a-7bdd-41cc-b180-8a7ca9db752e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:51.775593 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7a63a-7bdd-41cc-b180-8a7ca9db752e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:51.788590 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 326/652] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 18:59:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:51.789549 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7a63a-7bdd-41cc-b180-8a7ca9db752e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:51.821645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:51.832612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a08998-8b4f-4195-919f-d364d8326b07 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:51.868148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da1911-7db7-482b-918e-0d8820b20407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:51.869045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da1911-7db7-482b-918e-0d8820b20407 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:51.869664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da1911-7db7-482b-918e-0d8820b20407 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:51.870151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da1911-7db7-482b-918e-0d8820b20407 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:51.871582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da1911-7db7-482b-918e-0d8820b20407 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:51.872076 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da1911-7db7-482b-918e-0d8820b20407 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:51.873749 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da1911-7db7-482b-918e-0d8820b20407 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:51.893243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 327/653] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 31 18:59:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:51.894445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da1911-7db7-482b-918e-0d8820b20407 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:51.899314 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7a63a-7bdd-41cc-b180-8a7ca9db752e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:51.913146 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:51.939289 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fc9a7-682c-44ad-8bae-e87c636a3725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:51.940301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fc9a7-682c-44ad-8bae-e87c636a3725 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:51.940805 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fc9a7-682c-44ad-8bae-e87c636a3725 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:51.941777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411fc9a7-682c-44ad-8bae-e87c636a3725 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:51.942704 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fc9a7-682c-44ad-8bae-e87c636a3725 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:51.943217 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fc9a7-682c-44ad-8bae-e87c636a3725 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:51.945644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411fc9a7-682c-44ad-8bae-e87c636a3725 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:51.969984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 327/654] 10.222.0.65 () {56 vars in 1137 bytes} [Wed Aug 31 18:59:51 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:51.971159 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fc9a7-682c-44ad-8bae-e87c636a3725 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:52.024020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da1911-7db7-482b-918e-0d8820b20407 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:52.072767 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3d66b-fce2-446c-b1ed-624c19533cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:52.073742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3d66b-fce2-446c-b1ed-624c19533cfc None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:52.074944 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3d66b-fce2-446c-b1ed-624c19533cfc None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:52.075437 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3d66b-fce2-446c-b1ed-624c19533cfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:52.088242 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3d66b-fce2-446c-b1ed-624c19533cfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:52.089954 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3d66b-fce2-446c-b1ed-624c19533cfc None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:52.095222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411fc9a7-682c-44ad-8bae-e87c636a3725 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:52.106559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-ddd3d66b-fce2-446c-b1ed-624c19533cfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:52.106559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:52.106559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:52.106559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:52.106559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:52.106559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:52.106559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:52.106559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:52.106559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:52.106559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:52.106559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:52.106559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:52.106559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:52.106559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:52.106559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:52.106559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:52.107537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:52.108407 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:52.108407 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:52.108407 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:52.112426 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 328/655] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:52.113344 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3d66b-fce2-446c-b1ed-624c19533cfc None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:52.117491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:52.136362 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b6b26-98bb-42bc-9a14-1fdf1befc253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:52.137383 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b6b26-98bb-42bc-9a14-1fdf1befc253 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:52.137885 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b6b26-98bb-42bc-9a14-1fdf1befc253 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:52.138380 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b6b26-98bb-42bc-9a14-1fdf1befc253 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:52.185542 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:52.225303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b6b26-98bb-42bc-9a14-1fdf1befc253 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:52.225303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b6b26-98bb-42bc-9a14-1fdf1befc253 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:52.225562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3d66b-fce2-446c-b1ed-624c19533cfc None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:52.259228 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6e451-96e3-46b2-ade0-64142692af9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:52.259832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6e451-96e3-46b2-ade0-64142692af9c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:52.260106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6e451-96e3-46b2-ade0-64142692af9c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:52.260375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6e451-96e3-46b2-ade0-64142692af9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:52.319178 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 328/656] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:52.320039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b6b26-98bb-42bc-9a14-1fdf1befc253 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:52.332422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6e451-96e3-46b2-ade0-64142692af9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:52.333910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee6e451-96e3-46b2-ade0-64142692af9c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:52.335811 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:52.397614 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:52.425694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b6b26-98bb-42bc-9a14-1fdf1befc253 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:52.446017 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 329/657] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:52.447123 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6e451-96e3-46b2-ade0-64142692af9c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:52.469131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3e6cc-5b6b-42b4-8763-b623a88450dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:52.469865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3e6cc-5b6b-42b4-8763-b623a88450dd None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:52.470150 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3e6cc-5b6b-42b4-8763-b623a88450dd None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:52.470422 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f3e6cc-5b6b-42b4-8763-b623a88450dd None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:52.471063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3e6cc-5b6b-42b4-8763-b623a88450dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:52.471362 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3e6cc-5b6b-42b4-8763-b623a88450dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:52.473413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f3e6cc-5b6b-42b4-8763-b623a88450dd None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:52.490788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 329/658] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:52.491573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3e6cc-5b6b-42b4-8763-b623a88450dd None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:52.559802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6e451-96e3-46b2-ade0-64142692af9c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:52.561844 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:52.595602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f3e6cc-5b6b-42b4-8763-b623a88450dd None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:52.599311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a6d2c5-4c5d-4e82-8116-07608fadf432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:52.600191 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6d2c5-4c5d-4e82-8116-07608fadf432 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:52.600679 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6d2c5-4c5d-4e82-8116-07608fadf432 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:52.601167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a6d2c5-4c5d-4e82-8116-07608fadf432 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:52.612694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6d2c5-4c5d-4e82-8116-07608fadf432 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:52.614512 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a6d2c5-4c5d-4e82-8116-07608fadf432 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:52.624159 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:52.630047 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b66137-ecb3-467a-b989-d7527831caee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:52.630673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b66137-ecb3-467a-b989-d7527831caee None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:52.630950 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b66137-ecb3-467a-b989-d7527831caee None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:52.631218 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b66137-ecb3-467a-b989-d7527831caee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:52.632196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-a6a6d2c5-4c5d-4e82-8116-07608fadf432 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:52.632196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:52.632196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:52.632196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:52.632196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:52.632196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:52.632196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:52.632196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:52.632196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:52.632196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:52.632196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:52.632196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:52.632196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:52.632196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:52.632196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:52.632196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:52.633081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:52.634026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:52.634026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:52.634026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:52.634703 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 330/659] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:52.635551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49da39-e3ed-4408-81c0-d62ef868ee15 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:52.635827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49da39-e3ed-4408-81c0-d62ef868ee15 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:52.636096 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb49da39-e3ed-4408-81c0-d62ef868ee15 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:52.641565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 331/660] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:52.641565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49da39-e3ed-4408-81c0-d62ef868ee15 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:52.651025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b66137-ecb3-467a-b989-d7527831caee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:52.653154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b66137-ecb3-467a-b989-d7527831caee None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:52.667370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-84b66137-ecb3-467a-b989-d7527831caee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:52.667370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:52.667370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:52.667370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:52.667370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:52.667370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:52.667370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:52.667370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:52.667370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:52.667370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:52.667370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:52.667370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:52.667370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:52.667370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:52.667370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:52.667370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:52.669118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:52.670231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:52.670231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:52.670231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:52.670231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 330/661] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:52.670231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b66137-ecb3-467a-b989-d7527831caee None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:52.755912 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb49da39-e3ed-4408-81c0-d62ef868ee15 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:52.786021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b66137-ecb3-467a-b989-d7527831caee None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:52.791313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458329be-3007-40ec-9b1a-b232477f31fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:52.791932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458329be-3007-40ec-9b1a-b232477f31fa None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:52.792203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458329be-3007-40ec-9b1a-b232477f31fa None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:52.792465 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458329be-3007-40ec-9b1a-b232477f31fa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:52.800625 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458329be-3007-40ec-9b1a-b232477f31fa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:52.802073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458329be-3007-40ec-9b1a-b232477f31fa None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:52.813710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-458329be-3007-40ec-9b1a-b232477f31fa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 18:59:52.813710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:52.813710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:52.813710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:52.813710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:52.813710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:52.813710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:52.813710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:52.813710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:52.813710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:52.813710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:52.813710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:52.813710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 18:59:52.813710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 18:59:52.813710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 18:59:52.813710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:52.819476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:52.820513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:52.820513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:52.820513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 18:59:52.820513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 18:59:52.820513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 18:59:52.820513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 18:59:52.820513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 18:59:52.820513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 18:59:52.820513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 31 18:59:52.820513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:52.820513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 332/662] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:52.820513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7871234-81cd-43bd-89a6-ccb84ec4b970 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:52.820513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7871234-81cd-43bd-89a6-ccb84ec4b970 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:52.821269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7871234-81cd-43bd-89a6-ccb84ec4b970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:52.836282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a7871234-81cd-43bd-89a6-ccb84ec4b970 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:52.836913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f2781-8aab-435a-9d42-4ab6e515f311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:52.837455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f2781-8aab-435a-9d42-4ab6e515f311 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:52.837556 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a7871234-81cd-43bd-89a6-ccb84ec4b970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:52.837809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f2781-8aab-435a-9d42-4ab6e515f311 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:52.838155 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9f2781-8aab-435a-9d42-4ab6e515f311 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:52.838885 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f2781-8aab-435a-9d42-4ab6e515f311 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:52.839257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f2781-8aab-435a-9d42-4ab6e515f311 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:52.841289 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9f2781-8aab-435a-9d42-4ab6e515f311 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:52.848410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:52.854214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 331/663] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:52.855026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f2781-8aab-435a-9d42-4ab6e515f311 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:52.901946 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:52.914499 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 333/664] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:52.915445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7871234-81cd-43bd-89a6-ccb84ec4b970 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:52.991161 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9f2781-8aab-435a-9d42-4ab6e515f311 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:53.021704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7871234-81cd-43bd-89a6-ccb84ec4b970 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:53.038105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68114e86-b78c-4723-99ee-f80e99f53181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:53.039285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68114e86-b78c-4723-99ee-f80e99f53181 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:53.040127 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68114e86-b78c-4723-99ee-f80e99f53181 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:53.040600 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68114e86-b78c-4723-99ee-f80e99f53181 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:53.041454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68114e86-b78c-4723-99ee-f80e99f53181 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:53.041948 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68114e86-b78c-4723-99ee-f80e99f53181 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:53.044363 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68114e86-b78c-4723-99ee-f80e99f53181 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:53.057896 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51dacd-1c05-45a6-954c-529bff16c6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:53.058580 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51dacd-1c05-45a6-954c-529bff16c6f7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:53.058858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51dacd-1c05-45a6-954c-529bff16c6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:53.059180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb51dacd-1c05-45a6-954c-529bff16c6f7 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:53.059905 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51dacd-1c05-45a6-954c-529bff16c6f7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:53.060257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51dacd-1c05-45a6-954c-529bff16c6f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:53.061148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 332/665] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:53.062242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68114e86-b78c-4723-99ee-f80e99f53181 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:53.066697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb51dacd-1c05-45a6-954c-529bff16c6f7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:53.088086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 334/666] 10.222.0.65 () {56 vars in 1139 bytes} [Wed Aug 31 18:59:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:53.089281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51dacd-1c05-45a6-954c-529bff16c6f7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:53.093523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:53.172860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68114e86-b78c-4723-99ee-f80e99f53181 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:53.174528 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:53.200585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb51dacd-1c05-45a6-954c-529bff16c6f7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:53.208765 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969fe56-10fc-4141-8a85-9cddab1866ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:53.209425 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969fe56-10fc-4141-8a85-9cddab1866ed None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:53.209705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969fe56-10fc-4141-8a85-9cddab1866ed None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:53.209974 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969fe56-10fc-4141-8a85-9cddab1866ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:53.217227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969fe56-10fc-4141-8a85-9cddab1866ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:53.218639 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969fe56-10fc-4141-8a85-9cddab1866ed None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:53.229609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-2969fe56-10fc-4141-8a85-9cddab1866ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:53.229609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:53.229609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:53.229609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:53.229609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:53.229609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:53.229609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:53.229609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:53.229609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:53.229609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:53.229609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:53.229609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:53.229609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 18:59:53.229609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 18:59:53.229609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:53.230566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:53.231464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 18:59:53.231464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 18:59:53.231464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 18:59:53.231464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 18:59:53.231464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 18:59:53.231464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 18:59:53.231464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:53.231464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:53.231464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 333/667] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:53.231942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3459be-07ce-43b1-bf55-66ad45a9d3f4 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:53.232158 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3459be-07ce-43b1-bf55-66ad45a9d3f4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:53.232537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3459be-07ce-43b1-bf55-66ad45a9d3f4 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:53.237305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 334/668] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:53.237305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3459be-07ce-43b1-bf55-66ad45a9d3f4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:53.241309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c5c04-b623-4bd3-8a4b-2376a67e4828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:53.241943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c5c04-b623-4bd3-8a4b-2376a67e4828 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:53.242205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c5c04-b623-4bd3-8a4b-2376a67e4828 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:53.242470 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7c5c04-b623-4bd3-8a4b-2376a67e4828 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:53.249930 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c5c04-b623-4bd3-8a4b-2376a67e4828 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:53.251339 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7c5c04-b623-4bd3-8a4b-2376a67e4828 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:53.262672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-db7c5c04-b623-4bd3-8a4b-2376a67e4828 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:53.262672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:53.262672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:53.262672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:53.262672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:53.262672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:53.262672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:53.262672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:53.262672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:53.262672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:53.262672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:53.262672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:53.262672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 18:59:53.262672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 18:59:53.262672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:53.263643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:53.264527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 18:59:53.264527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 18:59:53.264527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 18:59:53.264527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 18:59:53.264527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 18:59:53.264527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 18:59:53.264527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:53.264527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:53.264527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 335/669] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:53.264527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c5c04-b623-4bd3-8a4b-2376a67e4828 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:53.325971 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:53.337311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3459be-07ce-43b1-bf55-66ad45a9d3f4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:53.364060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7c5c04-b623-4bd3-8a4b-2376a67e4828 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:53.369462 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ed6402-64bb-40d8-9bb4-a53744dac6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:53.370057 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed6402-64bb-40d8-9bb4-a53744dac6e4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:53.370330 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed6402-64bb-40d8-9bb4-a53744dac6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:53.370595 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ed6402-64bb-40d8-9bb4-a53744dac6e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:53.399741 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d03354-6ab5-46e0-bdbb-df92f487ad0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:53.400353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d03354-6ab5-46e0-bdbb-df92f487ad0b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:53.400568 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:53.400685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d03354-6ab5-46e0-bdbb-df92f487ad0b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:53.400869 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d03354-6ab5-46e0-bdbb-df92f487ad0b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:53.410810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d03354-6ab5-46e0-bdbb-df92f487ad0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:53.415924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-81d03354-6ab5-46e0-bdbb-df92f487ad0b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:59:53.415924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:53.415924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:53.415924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:53.415924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:53.415924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:53.415924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:53.415924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:53.415924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:53.415924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:53.415924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:53.415924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:53.415924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:53.415924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:53.415924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:53.415924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:53.417039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:53.417039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:53.417039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:53.417039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:53.417039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:53.417039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:53.417039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:53.417039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:53.417039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:53.417039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:53.417039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:53.417039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:53.417039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:53.417039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:53.417039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:53.417039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:53.418243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:53.418243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:53.418243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:53.418243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:53.418243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:53.418243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:53.418243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:53.418243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:53.418243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:53.418243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:53.418243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:59:53.418243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:53.418243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 336/670] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:59:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:53.418243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d03354-6ab5-46e0-bdbb-df92f487ad0b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:53.444722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed6402-64bb-40d8-9bb4-a53744dac6e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:53.446177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ed6402-64bb-40d8-9bb4-a53744dac6e4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:53.521027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d03354-6ab5-46e0-bdbb-df92f487ad0b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:53.543399 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:53.543655 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 335/671] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:53.544914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b08948-9455-4655-9146-f6e3776f9100 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:53.545233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b08948-9455-4655-9146-f6e3776f9100 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:53.545616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b08948-9455-4655-9146-f6e3776f9100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:53.558893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de2c7c-f42a-4cad-b0cd-f44590c9b20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:53.559504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de2c7c-f42a-4cad-b0cd-f44590c9b20e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:53.559776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de2c7c-f42a-4cad-b0cd-f44590c9b20e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:53.560041 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7de2c7c-f42a-4cad-b0cd-f44590c9b20e None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:53.560714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de2c7c-f42a-4cad-b0cd-f44590c9b20e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:53.561007 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de2c7c-f42a-4cad-b0cd-f44590c9b20e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:53.562442 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7de2c7c-f42a-4cad-b0cd-f44590c9b20e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:53.563048 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-14b08948-9455-4655-9146-f6e3776f9100 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:53.564210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-14b08948-9455-4655-9146-f6e3776f9100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:53.574779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 337/672] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 18:59:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:53.575555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de2c7c-f42a-4cad-b0cd-f44590c9b20e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:53.620773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:53.631886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 336/673] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:53.632687 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b08948-9455-4655-9146-f6e3776f9100 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:53.680875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7de2c7c-f42a-4cad-b0cd-f44590c9b20e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:53.717511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba64d8c-1608-45b1-9392-8f1acb98f023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:53.718228 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba64d8c-1608-45b1-9392-8f1acb98f023 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:53.718503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba64d8c-1608-45b1-9392-8f1acb98f023 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:53.718771 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba64d8c-1608-45b1-9392-8f1acb98f023 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:53.719449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba64d8c-1608-45b1-9392-8f1acb98f023 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:53.719748 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba64d8c-1608-45b1-9392-8f1acb98f023 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:53.721411 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba64d8c-1608-45b1-9392-8f1acb98f023 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:53.729074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b08948-9455-4655-9146-f6e3776f9100 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:53.737242 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 338/674] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 18:59:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:53.738337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba64d8c-1608-45b1-9392-8f1acb98f023 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:53.765831 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a5cfb-4543-4ebf-8926-99c9acdaa150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:53.766426 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a5cfb-4543-4ebf-8926-99c9acdaa150 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:53.766701 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a5cfb-4543-4ebf-8926-99c9acdaa150 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:53.766972 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a5cfb-4543-4ebf-8926-99c9acdaa150 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:53.767623 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a5cfb-4543-4ebf-8926-99c9acdaa150 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:53.767920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a5cfb-4543-4ebf-8926-99c9acdaa150 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:53.769515 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339a5cfb-4543-4ebf-8926-99c9acdaa150 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:53.782718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 337/675] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:53.782718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a5cfb-4543-4ebf-8926-99c9acdaa150 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:53.787249 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:53.840567 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:53.848803 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba64d8c-1608-45b1-9392-8f1acb98f023 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:53.886306 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505aa8f-be45-4c69-8fd5-35ceadf2082e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:53.887027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505aa8f-be45-4c69-8fd5-35ceadf2082e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:53.887284 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a5cfb-4543-4ebf-8926-99c9acdaa150 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:53.887388 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505aa8f-be45-4c69-8fd5-35ceadf2082e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:53.887561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d505aa8f-be45-4c69-8fd5-35ceadf2082e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:53.898189 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505aa8f-be45-4c69-8fd5-35ceadf2082e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:53.899593 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d505aa8f-be45-4c69-8fd5-35ceadf2082e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:53.913533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-d505aa8f-be45-4c69-8fd5-35ceadf2082e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:53.913533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:53.913533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:53.913533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:53.913533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:53.913533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:53.913533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:53.913533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:53.913533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:53.913533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:53.913533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:53.913533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:53.913533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:53.913533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:53.913533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:53.913533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:53.919824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:53.920835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:53.920835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:53.920835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:53.920835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 339/676] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:53.920835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505aa8f-be45-4c69-8fd5-35ceadf2082e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:53.922050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1103f0-047f-4a4d-98a3-02b0f2bdb908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:53.922643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1103f0-047f-4a4d-98a3-02b0f2bdb908 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:53.922915 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1103f0-047f-4a4d-98a3-02b0f2bdb908 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:53.923177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1103f0-047f-4a4d-98a3-02b0f2bdb908 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:53.923887 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1103f0-047f-4a4d-98a3-02b0f2bdb908 None admin] RBAC: Adding query filter params (role.id=6e93fc836fe74c9daa11c7b2848a2922, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=5d157abcf94745a5b92d6df64d872f73) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:53.924186 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1103f0-047f-4a4d-98a3-02b0f2bdb908 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:53.925619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1103f0-047f-4a4d-98a3-02b0f2bdb908 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:53.931222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 338/677] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:59:53 2022] GET /identity/v3/role_assignments?user.id=5d157abcf94745a5b92d6df64d872f73&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=6e93fc836fe74c9daa11c7b2848a2922 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:53.931973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1103f0-047f-4a4d-98a3-02b0f2bdb908 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:54.021902 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:54.030617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d505aa8f-be45-4c69-8fd5-35ceadf2082e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:54.066107 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1103f0-047f-4a4d-98a3-02b0f2bdb908 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:54.069351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174372e-ed2d-4749-9dd3-4e330b5b4f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:54.069956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174372e-ed2d-4749-9dd3-4e330b5b4f93 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:54.070229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174372e-ed2d-4749-9dd3-4e330b5b4f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:54.070492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174372e-ed2d-4749-9dd3-4e330b5b4f93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:54.071211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174372e-ed2d-4749-9dd3-4e330b5b4f93 None admin] RBAC: Adding query filter params (role.id=6e93fc836fe74c9daa11c7b2848a2922, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=1863336fecad4987acdfd0b3f7b2f70f) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:54.071509 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174372e-ed2d-4749-9dd3-4e330b5b4f93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:54.073729 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174372e-ed2d-4749-9dd3-4e330b5b4f93 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:54.080056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 340/678] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:59:53 2022] GET /identity/v3/role_assignments?user.id=1863336fecad4987acdfd0b3f7b2f70f&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=6e93fc836fe74c9daa11c7b2848a2922 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:54.080813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174372e-ed2d-4749-9dd3-4e330b5b4f93 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:54.094280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:54.120182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b12d4c-8ac1-4cd4-ab7e-9e93c251eb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:54.120182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b12d4c-8ac1-4cd4-ab7e-9e93c251eb96 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:54.120182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b12d4c-8ac1-4cd4-ab7e-9e93c251eb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:54.120182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b12d4c-8ac1-4cd4-ab7e-9e93c251eb96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:54.132996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b12d4c-8ac1-4cd4-ab7e-9e93c251eb96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:54.134526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b12d4c-8ac1-4cd4-ab7e-9e93c251eb96 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:54.151404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-b1b12d4c-8ac1-4cd4-ab7e-9e93c251eb96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:54.151404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:54.151404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:54.151404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:54.151404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:54.151404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:54.151404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:54.151404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:54.151404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:54.151404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:54.151404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:54.151404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:54.151404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:54.151404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:54.151404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:54.151404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:54.152916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:54.154013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:54.154013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:54.154013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:54.154013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 339/679] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:54.154013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b12d4c-8ac1-4cd4-ab7e-9e93c251eb96 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:54.251750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174372e-ed2d-4749-9dd3-4e330b5b4f93 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:54.294133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b12d4c-8ac1-4cd4-ab7e-9e93c251eb96 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:54.305401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb9fb8-9342-43bd-a1c4-fa6f98a08528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:54.306116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb9fb8-9342-43bd-a1c4-fa6f98a08528 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:54.306506 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb9fb8-9342-43bd-a1c4-fa6f98a08528 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:54.306777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb9fb8-9342-43bd-a1c4-fa6f98a08528 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:54.322236 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb9fb8-9342-43bd-a1c4-fa6f98a08528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:54.328660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-40eb9fb8-9342-43bd-a1c4-fa6f98a08528 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:54.328660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:54.328660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:54.328660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:54.328660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:54.328660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:54.328660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:54.328660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:54.328660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:54.328660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:54.328660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:54.328660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:54.328660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:54.328660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:54.328660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:54.328660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:54.329655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:54.329655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:54.329655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:54.329655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:54.329655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:54.329655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:54.329655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:54.329655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:54.329655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:54.329655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:54.329655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:54.329655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:54.329655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:54.329655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:54.329655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:54.329655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:54.330516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:54.330516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:54.330516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:54.330516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:54.330516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:54.330516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:54.330516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:54.330516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:54.330516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:54.330516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:54.330516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 18:59:54.330516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:54.330516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 341/680] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 18:59:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:54.331206 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb9fb8-9342-43bd-a1c4-fa6f98a08528 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:54.343102 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c42ca-4573-4b5e-a448-922354749a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:54.343728 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c42ca-4573-4b5e-a448-922354749a10 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:54.344010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c42ca-4573-4b5e-a448-922354749a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:54.344280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5c42ca-4573-4b5e-a448-922354749a10 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:54.344928 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c42ca-4573-4b5e-a448-922354749a10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:54.345252 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c42ca-4573-4b5e-a448-922354749a10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:54.346627 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5c42ca-4573-4b5e-a448-922354749a10 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:54.363540 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 340/681] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:54.364407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c42ca-4573-4b5e-a448-922354749a10 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:54.365366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:54.420542 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:54.467152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb9fb8-9342-43bd-a1c4-fa6f98a08528 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:54.504671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c418b1-6de9-444c-96a0-c7918069c9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:54.504950 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c418b1-6de9-444c-96a0-c7918069c9d7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:54.505348 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c418b1-6de9-444c-96a0-c7918069c9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:54.505663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c418b1-6de9-444c-96a0-c7918069c9d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:54.506305 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c418b1-6de9-444c-96a0-c7918069c9d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:54.506607 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c418b1-6de9-444c-96a0-c7918069c9d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:54.507971 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c418b1-6de9-444c-96a0-c7918069c9d7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:54.521391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 342/682] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:54.522306 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c418b1-6de9-444c-96a0-c7918069c9d7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:54.531788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5c42ca-4573-4b5e-a448-922354749a10 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:54.578158 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee9350-46d0-435a-a83b-1dc961d3efe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:54.578768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee9350-46d0-435a-a83b-1dc961d3efe6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:54.579041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee9350-46d0-435a-a83b-1dc961d3efe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:54.579317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee9350-46d0-435a-a83b-1dc961d3efe6 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:54.580376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee9350-46d0-435a-a83b-1dc961d3efe6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:54.580754 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee9350-46d0-435a-a83b-1dc961d3efe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:54.582736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee9350-46d0-435a-a83b-1dc961d3efe6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:54.596430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 341/683] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 18:59:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:54.596430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee9350-46d0-435a-a83b-1dc961d3efe6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:54.628789 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:54.660501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c418b1-6de9-444c-96a0-c7918069c9d7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:54.703529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c943db-e343-4c5b-a830-a59e56e64247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:54.704334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c943db-e343-4c5b-a830-a59e56e64247 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:54.704716 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c943db-e343-4c5b-a830-a59e56e64247 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:54.705138 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c943db-e343-4c5b-a830-a59e56e64247 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:54.706226 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:54.720212 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c943db-e343-4c5b-a830-a59e56e64247 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:54.722109 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c943db-e343-4c5b-a830-a59e56e64247 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:54.723455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee9350-46d0-435a-a83b-1dc961d3efe6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:54.735081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-57c943db-e343-4c5b-a830-a59e56e64247 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:54.735081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:54.735081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:54.735081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:54.735081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:54.735081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:54.735081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:54.735081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:54.735081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:54.735081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:54.735081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:54.735081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:54.735081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 18:59:54.735081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 18:59:54.735081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:54.736171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:54.737245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 18:59:54.737245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 18:59:54.737245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 18:59:54.737245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 18:59:54.737245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 18:59:54.737245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 18:59:54.737245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:54.737245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:54.738459 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 343/684] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:54.739313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c943db-e343-4c5b-a830-a59e56e64247 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:54.772319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857c81d-a990-4180-abfb-344300bd9c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:54.773456 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857c81d-a990-4180-abfb-344300bd9c70 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:54.773861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857c81d-a990-4180-abfb-344300bd9c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:54.774136 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857c81d-a990-4180-abfb-344300bd9c70 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:54.784944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857c81d-a990-4180-abfb-344300bd9c70 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:54.787622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857c81d-a990-4180-abfb-344300bd9c70 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:54.802208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-e857c81d-a990-4180-abfb-344300bd9c70 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 18:59:54.802208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:54.802208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:54.802208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:54.802208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:54.802208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:54.802208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:54.802208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:54.802208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:54.802208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:54.802208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:54.802208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:54.802208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 18:59:54.802208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 18:59:54.802208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 18:59:54.802208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:54.803231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:54.804141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:54.804141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:54.804141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 18:59:54.804141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 18:59:54.804141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 18:59:54.804141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 18:59:54.804141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 18:59:54.804141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 18:59:54.804141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 18:59:54.804141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:54.809546 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 342/685] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 18:59:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:54.810540 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857c81d-a990-4180-abfb-344300bd9c70 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:54.868649 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c943db-e343-4c5b-a830-a59e56e64247 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:54.911552 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87043d-8d40-4aff-9dc9-4606b1775f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:54.912184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87043d-8d40-4aff-9dc9-4606b1775f64 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:54.912463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87043d-8d40-4aff-9dc9-4606b1775f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:54.912833 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef87043d-8d40-4aff-9dc9-4606b1775f64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:54.919439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857c81d-a990-4180-abfb-344300bd9c70 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:54.920846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:54.925257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87043d-8d40-4aff-9dc9-4606b1775f64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:54.927057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef87043d-8d40-4aff-9dc9-4606b1775f64 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:54.944834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-ef87043d-8d40-4aff-9dc9-4606b1775f64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:54.944834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:54.944834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:54.944834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:54.944834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:54.944834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:54.944834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:54.944834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:54.944834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:54.944834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:54.944834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:54.944834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:54.944834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:54.944834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:54.944834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:54.944834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:54.947135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:54.949914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:54.949914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:54.949914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:54.949914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 344/686] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:54.949914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87043d-8d40-4aff-9dc9-4606b1775f64 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:54.968061 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446ce1d-1ded-457e-84b3-b9d9c431ae8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:54.968861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446ce1d-1ded-457e-84b3-b9d9c431ae8c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:54.969169 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446ce1d-1ded-457e-84b3-b9d9c431ae8c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:54.969943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446ce1d-1ded-457e-84b3-b9d9c431ae8c None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:54.970704 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446ce1d-1ded-457e-84b3-b9d9c431ae8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:54.971017 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446ce1d-1ded-457e-84b3-b9d9c431ae8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:54.972439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446ce1d-1ded-457e-84b3-b9d9c431ae8c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:54.996531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 343/687] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 18:59:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:54.997579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446ce1d-1ded-457e-84b3-b9d9c431ae8c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:55.017769 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:55.075500 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef87043d-8d40-4aff-9dc9-4606b1775f64 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:55.112074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d0115-e26e-4310-8dd8-ef9f7e33ddc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:55.112350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d0115-e26e-4310-8dd8-ef9f7e33ddc0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:55.112634 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d0115-e26e-4310-8dd8-ef9f7e33ddc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:55.112905 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d0115-e26e-4310-8dd8-ef9f7e33ddc0 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:55.113775 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d0115-e26e-4310-8dd8-ef9f7e33ddc0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:55.114005 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d0115-e26e-4310-8dd8-ef9f7e33ddc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:55.115794 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787d0115-e26e-4310-8dd8-ef9f7e33ddc0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:55.140863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 345/688] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 18:59:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:55.140863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d0115-e26e-4310-8dd8-ef9f7e33ddc0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:55.142294 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446ce1d-1ded-457e-84b3-b9d9c431ae8c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:55.197327 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5333a12-e9b1-4676-9a16-9d5cd7704ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:55.198463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5333a12-e9b1-4676-9a16-9d5cd7704ada None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:55.199012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5333a12-e9b1-4676-9a16-9d5cd7704ada None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:55.199494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5333a12-e9b1-4676-9a16-9d5cd7704ada None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:55.200964 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5333a12-e9b1-4676-9a16-9d5cd7704ada None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:55.201517 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5333a12-e9b1-4676-9a16-9d5cd7704ada None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:55.204037 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5333a12-e9b1-4676-9a16-9d5cd7704ada None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:55.216138 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:55.229691 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 344/689] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:55.231483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5333a12-e9b1-4676-9a16-9d5cd7704ada None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:55.267807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d0115-e26e-4310-8dd8-ef9f7e33ddc0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:55.306377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee95a2-e162-49ed-8d75-70ff7465c137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:55.306983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee95a2-e162-49ed-8d75-70ff7465c137 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:55.307258 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee95a2-e162-49ed-8d75-70ff7465c137 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:55.307523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee95a2-e162-49ed-8d75-70ff7465c137 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:55.308229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee95a2-e162-49ed-8d75-70ff7465c137 None admin] RBAC: Adding query filter params (role.id=6e93fc836fe74c9daa11c7b2848a2922, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=093edc8c192c469f9532a45bb3326612) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:55.309275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee95a2-e162-49ed-8d75-70ff7465c137 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:55.310709 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee95a2-e162-49ed-8d75-70ff7465c137 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:55.316737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 346/690] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:59:55 2022] GET /identity/v3/role_assignments?user.id=093edc8c192c469f9532a45bb3326612&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=6e93fc836fe74c9daa11c7b2848a2922 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:55.325229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee95a2-e162-49ed-8d75-70ff7465c137 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:55.344097 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:55.402105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5333a12-e9b1-4676-9a16-9d5cd7704ada None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:55.482330 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3142208-0ef3-48ba-895a-06c73eb25a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:55.482874 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3142208-0ef3-48ba-895a-06c73eb25a55 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:55.483149 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3142208-0ef3-48ba-895a-06c73eb25a55 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:55.483416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3142208-0ef3-48ba-895a-06c73eb25a55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:55.546876 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee95a2-e162-49ed-8d75-70ff7465c137 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:55.573291 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:55.602165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3142208-0ef3-48ba-895a-06c73eb25a55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:55.604943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3142208-0ef3-48ba-895a-06c73eb25a55 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:55.608222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33912cc7-5108-4577-b22d-0ace7589820b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:55.609916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33912cc7-5108-4577-b22d-0ace7589820b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:55.610295 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33912cc7-5108-4577-b22d-0ace7589820b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:55.610662 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33912cc7-5108-4577-b22d-0ace7589820b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:55.619867 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33912cc7-5108-4577-b22d-0ace7589820b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:55.621875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33912cc7-5108-4577-b22d-0ace7589820b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:55.637278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-33912cc7-5108-4577-b22d-0ace7589820b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 18:59:55.637278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:55.637278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:55.637278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:55.637278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:55.637278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:55.637278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:55.637278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:55.637278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:55.637278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:55.637278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:55.637278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:55.637278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 18:59:55.637278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 18:59:55.637278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 18:59:55.637278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:55.638320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:55.639287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:55.639287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:55.639287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 18:59:55.639287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 18:59:55.639287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 18:59:55.639287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 18:59:55.639287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 18:59:55.639287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 18:59:55.639287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 18:59:55.639287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:55.639287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 347/691] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:59:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:55.644547 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33912cc7-5108-4577-b22d-0ace7589820b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:55.700340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:55.753768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 345/692] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 31 18:59:55.756501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3142208-0ef3-48ba-895a-06c73eb25a55 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:55.823400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33912cc7-5108-4577-b22d-0ace7589820b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:55.878729 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3142208-0ef3-48ba-895a-06c73eb25a55 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:55.902060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3895081-139a-4144-b00b-d15fc89a4ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:55.903283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3895081-139a-4144-b00b-d15fc89a4ffb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:55.904593 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3895081-139a-4144-b00b-d15fc89a4ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:55.904593 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3895081-139a-4144-b00b-d15fc89a4ffb None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:55.904593 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3895081-139a-4144-b00b-d15fc89a4ffb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:55.904845 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3895081-139a-4144-b00b-d15fc89a4ffb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:55.906227 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3895081-139a-4144-b00b-d15fc89a4ffb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:55.932733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc3036-1512-4df3-8f9c-d06b54828e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:55.933426 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc3036-1512-4df3-8f9c-d06b54828e12 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:55.933701 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc3036-1512-4df3-8f9c-d06b54828e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:55.933966 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc3036-1512-4df3-8f9c-d06b54828e12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:55.944858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc3036-1512-4df3-8f9c-d06b54828e12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:55.945055 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 348/693] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 31 18:59:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:55.946319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fc3036-1512-4df3-8f9c-d06b54828e12 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:55.952612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3895081-139a-4144-b00b-d15fc89a4ffb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:55.967581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-b0fc3036-1512-4df3-8f9c-d06b54828e12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:55.967581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:55.967581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:55.967581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:55.967581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:55.967581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:55.967581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:55.967581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:55.967581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:55.967581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:55.967581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:55.967581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:55.967581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:55.967581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:55.967581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:55.967581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:55.969044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:55.969987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:55.969987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:55.969987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:55.969987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 346/694] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 18:59:55.979459 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c0ca7-d207-4ead-8ac3-c93ad093a91f None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:55.979736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c0ca7-d207-4ead-8ac3-c93ad093a91f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:55.980261 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c0ca7-d207-4ead-8ac3-c93ad093a91f None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:55.985275 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 347/695] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:55.986120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c0ca7-d207-4ead-8ac3-c93ad093a91f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:55.996674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:56.016216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:56.070413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3895081-139a-4144-b00b-d15fc89a4ffb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:56.103915 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c0ca7-d207-4ead-8ac3-c93ad093a91f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:56.105667 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee2894-4767-4319-86d8-b70444237707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:56.106458 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee2894-4767-4319-86d8-b70444237707 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:56.106730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee2894-4767-4319-86d8-b70444237707 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:56.107053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee2894-4767-4319-86d8-b70444237707 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:56.145959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ccaee9-da87-4c24-a395-1a2b560c635a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:56.146932 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccaee9-da87-4c24-a395-1a2b560c635a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:56.147413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccaee9-da87-4c24-a395-1a2b560c635a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:56.148267 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ccaee9-da87-4c24-a395-1a2b560c635a None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:56.149680 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ccaee9-da87-4c24-a395-1a2b560c635a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:56.150232 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ccaee9-da87-4c24-a395-1a2b560c635a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:56.153279 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ccaee9-da87-4c24-a395-1a2b560c635a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:56.171474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 348/696] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:56.172851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d44ce-d222-4fa5-8025-a7e5a7ae7a29 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:56.177600 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d44ce-d222-4fa5-8025-a7e5a7ae7a29 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:56.177889 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609d44ce-d222-4fa5-8025-a7e5a7ae7a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:56.186260 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee2894-4767-4319-86d8-b70444237707 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:56.187753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee2894-4767-4319-86d8-b70444237707 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:56.205067 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-609d44ce-d222-4fa5-8025-a7e5a7ae7a29 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:56.206652 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-609d44ce-d222-4fa5-8025-a7e5a7ae7a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:56.218506 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:56.289111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 349/697] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:56.290251 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d44ce-d222-4fa5-8025-a7e5a7ae7a29 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:56.296298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 349/698] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 18:59:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 31 18:59:56.297213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee2894-4767-4319-86d8-b70444237707 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:56.307478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:56.398827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee2894-4767-4319-86d8-b70444237707 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:56.427048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:56.433155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb1411-d0ac-4405-b6b4-4903490a4da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:56.434126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609d44ce-d222-4fa5-8025-a7e5a7ae7a29 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:56.434603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb1411-d0ac-4405-b6b4-4903490a4da5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:56.435194 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb1411-d0ac-4405-b6b4-4903490a4da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:56.435525 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb1411-d0ac-4405-b6b4-4903490a4da5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:56.445826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb1411-d0ac-4405-b6b4-4903490a4da5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:56.447339 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb1411-d0ac-4405-b6b4-4903490a4da5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:56.462410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-10eb1411-d0ac-4405-b6b4-4903490a4da5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:56.462410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:56.462410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:56.462410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:56.462410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:56.462410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:56.462410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:56.462410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:56.462410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:56.462410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:56.462410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:56.462410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:56.462410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:56.462410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:56.462410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:56.462410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:56.463544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:56.464688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:56.464688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:56.464688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:56.471219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 350/699] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:56.471219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d78066-34d3-44a5-96c0-8e20bdf437d0 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:56.471476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d78066-34d3-44a5-96c0-8e20bdf437d0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:56.471878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d78066-34d3-44a5-96c0-8e20bdf437d0 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:56.473307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 351/700] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:56.473820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d78066-34d3-44a5-96c0-8e20bdf437d0 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:56.482803 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648bedf9-5bba-426c-8a89-73f3333a85c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:56.483466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bedf9-5bba-426c-8a89-73f3333a85c9 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:56.483747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bedf9-5bba-426c-8a89-73f3333a85c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:56.484020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648bedf9-5bba-426c-8a89-73f3333a85c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:56.493093 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bedf9-5bba-426c-8a89-73f3333a85c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:56.494919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648bedf9-5bba-426c-8a89-73f3333a85c9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:56.511153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-648bedf9-5bba-426c-8a89-73f3333a85c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:56.511153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:56.511153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:56.511153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:56.511153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:56.511153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:56.511153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:56.511153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:56.511153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:56.511153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:56.511153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:56.511153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:56.511153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 18:59:56.511153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 18:59:56.511153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:56.512616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:56.513619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 18:59:56.513619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 18:59:56.513619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 18:59:56.513619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 18:59:56.513619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 18:59:56.513619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 18:59:56.513619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:56.513619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:56.514292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 350/701] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:56.515316 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b09d8-864b-4acf-a690-a9a1323ed01b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:56.515506 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b09d8-864b-4acf-a690-a9a1323ed01b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:56.517340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b09d8-864b-4acf-a690-a9a1323ed01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:56.535854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-426b09d8-864b-4acf-a690-a9a1323ed01b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:56.537167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-426b09d8-864b-4acf-a690-a9a1323ed01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:56.593088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d78066-34d3-44a5-96c0-8e20bdf437d0 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:56.612202 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 351/702] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:56.613049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b09d8-864b-4acf-a690-a9a1323ed01b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:56.622088 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:56.639136 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9613922-5165-4400-8d04-16eb9c1aba30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:56.640546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9613922-5165-4400-8d04-16eb9c1aba30 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:56.641005 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9613922-5165-4400-8d04-16eb9c1aba30 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:56.641743 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9613922-5165-4400-8d04-16eb9c1aba30 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:56.642561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9613922-5165-4400-8d04-16eb9c1aba30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:56.643231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9613922-5165-4400-8d04-16eb9c1aba30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:56.645488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9613922-5165-4400-8d04-16eb9c1aba30 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:56.661780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 352/703] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:56.663293 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9613922-5165-4400-8d04-16eb9c1aba30 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:56.723578 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:56.728755 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b09d8-864b-4acf-a690-a9a1323ed01b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:56.766454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9c0ec-99ca-4ab7-9fa0-ab24b1cdba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:56.767224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9c0ec-99ca-4ab7-9fa0-ab24b1cdba7c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:56.767684 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9c0ec-99ca-4ab7-9fa0-ab24b1cdba7c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:56.768051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9c0ec-99ca-4ab7-9fa0-ab24b1cdba7c None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:56.769036 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9c0ec-99ca-4ab7-9fa0-ab24b1cdba7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:56.769285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9c0ec-99ca-4ab7-9fa0-ab24b1cdba7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:56.770785 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9c0ec-99ca-4ab7-9fa0-ab24b1cdba7c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:56.789399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 352/704] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 18:59:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:56.789726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9613922-5165-4400-8d04-16eb9c1aba30 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:56.790523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9c0ec-99ca-4ab7-9fa0-ab24b1cdba7c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:56.829678 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4118e-aba2-4d3d-b83f-dfe0c6ee6cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:56.830737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4118e-aba2-4d3d-b83f-dfe0c6ee6cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:56.831225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4118e-aba2-4d3d-b83f-dfe0c6ee6cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:56.831695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4118e-aba2-4d3d-b83f-dfe0c6ee6cf4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:56.832469 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4118e-aba2-4d3d-b83f-dfe0c6ee6cf4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:56.834083 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b4118e-aba2-4d3d-b83f-dfe0c6ee6cf4 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-d6b4118e-aba2-4d3d-b83f-dfe0c6ee6cf4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:56.845737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:56.846746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:56.846746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:56.846746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:56.846746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:56.846746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:56.846746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:56.846746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 18:59:56.846746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 18:59:56.846746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 18:59:56.846746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 18:59:56.846746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:56.846746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:56.846746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:56.846746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:56.846746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:56.846746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:56.847596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:56.847596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:56.847596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:56.847596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:56.847596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:56.847596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:56.847596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:56.847596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:56.847596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:56.847596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:56.847596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 18:59:56.847596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 18:59:56.847596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 18:59:56.847596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:56.850326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 353/705] 10.222.0.65 () {56 vars in 1129 bytes} [Wed Aug 31 18:59:56 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:56.851323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4118e-aba2-4d3d-b83f-dfe0c6ee6cf4 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:56.864374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:56.906742 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9c0ec-99ca-4ab7-9fa0-ab24b1cdba7c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:56.945683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b2f22-5f96-4bff-a367-1fac955c8279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:56.946305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b2f22-5f96-4bff-a367-1fac955c8279 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:56.946581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b2f22-5f96-4bff-a367-1fac955c8279 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:56.946849 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b2f22-5f96-4bff-a367-1fac955c8279 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:56.953583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b2f22-5f96-4bff-a367-1fac955c8279 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:56.954999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b2f22-5f96-4bff-a367-1fac955c8279 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:56.963359 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b4118e-aba2-4d3d-b83f-dfe0c6ee6cf4 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:56.966521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-4c3b2f22-5f96-4bff-a367-1fac955c8279 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:56.966521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:56.966521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:56.966521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:56.966521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:56.966521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:56.966521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:56.966521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:56.966521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:56.966521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:56.966521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:56.966521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:56.966521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 18:59:56.966521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 18:59:56.966521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:56.967386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:56.968677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 18:59:56.968677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 18:59:56.968677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 18:59:56.968677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 18:59:56.968677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 18:59:56.968677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 18:59:56.968677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 18:59:56.968677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:56.968677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 353/706] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:56.968677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b2f22-5f96-4bff-a367-1fac955c8279 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:56.979183 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:57.009670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b04b9a-dfea-4d00-b492-c2e3783e88b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:57.010338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b04b9a-dfea-4d00-b492-c2e3783e88b9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:57.010613 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b04b9a-dfea-4d00-b492-c2e3783e88b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:57.010876 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b04b9a-dfea-4d00-b492-c2e3783e88b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:57.011605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b04b9a-dfea-4d00-b492-c2e3783e88b9 None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=673a72ac9a9847e3812007b66ac7a5ea) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:57.011903 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b04b9a-dfea-4d00-b492-c2e3783e88b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:57.013753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b04b9a-dfea-4d00-b492-c2e3783e88b9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:57.019875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 354/707] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:59:56 2022] GET /identity/v3/role_assignments?user.id=673a72ac9a9847e3812007b66ac7a5ea&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=3580c894dd974d35bc95848b8ba22598 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:57.020579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b04b9a-dfea-4d00-b492-c2e3783e88b9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:57.093358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b2f22-5f96-4bff-a367-1fac955c8279 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:57.121139 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:57.134327 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f6d1d-7651-4f06-8c51-04b4e7bb110c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:57.135062 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f6d1d-7651-4f06-8c51-04b4e7bb110c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:57.135413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f6d1d-7651-4f06-8c51-04b4e7bb110c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:57.135755 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f6d1d-7651-4f06-8c51-04b4e7bb110c None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:57.136514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f6d1d-7651-4f06-8c51-04b4e7bb110c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:57.136880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f6d1d-7651-4f06-8c51-04b4e7bb110c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:57.138546 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f6d1d-7651-4f06-8c51-04b4e7bb110c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:57.148084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 354/708] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 31 18:59:56 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:57.148084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f6d1d-7651-4f06-8c51-04b4e7bb110c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:57.180094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b04b9a-dfea-4d00-b492-c2e3783e88b9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:57.233218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446600b-b949-4663-9699-07b10e04edf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:57.233882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446600b-b949-4663-9699-07b10e04edf7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:57.234213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446600b-b949-4663-9699-07b10e04edf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:57.234490 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446600b-b949-4663-9699-07b10e04edf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:57.245912 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446600b-b949-4663-9699-07b10e04edf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:57.247389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446600b-b949-4663-9699-07b10e04edf7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:57.262488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-b446600b-b949-4663-9699-07b10e04edf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:57.262488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:57.262488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:57.262488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:57.262488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:57.262488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:57.262488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:57.262488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:57.262488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:57.262488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:57.262488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:57.262488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:57.262488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 18:59:57.262488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 18:59:57.262488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 18:59:57.262488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:57.263468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 18:59:57.264403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 18:59:57.264403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 18:59:57.264403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:57.264403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 355/709] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 18:59:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:57.265114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446600b-b949-4663-9699-07b10e04edf7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:57.276042 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f6d1d-7651-4f06-8c51-04b4e7bb110c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:57.290993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:57.314751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f06e7-2767-44cb-ad03-d4150aa69a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:57.315815 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f06e7-2767-44cb-ad03-d4150aa69a39 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:57.316299 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f06e7-2767-44cb-ad03-d4150aa69a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:57.316773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f06e7-2767-44cb-ad03-d4150aa69a39 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:57.317718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f06e7-2767-44cb-ad03-d4150aa69a39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:57.318238 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f06e7-2767-44cb-ad03-d4150aa69a39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:57.320179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3f06e7-2767-44cb-ad03-d4150aa69a39 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:57.334917 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 355/710] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 18:59:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:57.336158 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f06e7-2767-44cb-ad03-d4150aa69a39 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:57.379765 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446600b-b949-4663-9699-07b10e04edf7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:57.395232 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:57.415367 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598822f5-a601-48d1-97d5-5802c3340c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:57.415978 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598822f5-a601-48d1-97d5-5802c3340c3c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:57.416255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598822f5-a601-48d1-97d5-5802c3340c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:57.416526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598822f5-a601-48d1-97d5-5802c3340c3c None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:57.417174 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598822f5-a601-48d1-97d5-5802c3340c3c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:57.417869 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598822f5-a601-48d1-97d5-5802c3340c3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:57.419073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598822f5-a601-48d1-97d5-5802c3340c3c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:57.433217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 356/711] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 31 18:59:57 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:57.433217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598822f5-a601-48d1-97d5-5802c3340c3c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:57.476818 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f06e7-2767-44cb-ad03-d4150aa69a39 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:57.528977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:57.562741 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7682d09-d2ed-4cab-a225-df8378181729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:57.563370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7682d09-d2ed-4cab-a225-df8378181729 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:57.563652 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7682d09-d2ed-4cab-a225-df8378181729 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:57.563924 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7682d09-d2ed-4cab-a225-df8378181729 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:57.564632 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7682d09-d2ed-4cab-a225-df8378181729 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:57.564943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7682d09-d2ed-4cab-a225-df8378181729 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:57.567284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598822f5-a601-48d1-97d5-5802c3340c3c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:57.571106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7682d09-d2ed-4cab-a225-df8378181729 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:57.598460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 356/712] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 18:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:57.606383 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7682d09-d2ed-4cab-a225-df8378181729 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:57.656326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa48052-4788-4453-83ca-63d1b6d1180b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:57.656866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa48052-4788-4453-83ca-63d1b6d1180b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:57.657133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa48052-4788-4453-83ca-63d1b6d1180b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:57.657432 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa48052-4788-4453-83ca-63d1b6d1180b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:57.658161 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa48052-4788-4453-83ca-63d1b6d1180b None admin] RBAC: Adding query filter params (role.id=6e93fc836fe74c9daa11c7b2848a2922, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=586715d11b924e2cb6f44030efd77619) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:57.658456 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa48052-4788-4453-83ca-63d1b6d1180b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:57.659869 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa48052-4788-4453-83ca-63d1b6d1180b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:57.665829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 357/713] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 18:59:57 2022] GET /identity/v3/role_assignments?user.id=586715d11b924e2cb6f44030efd77619&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=6e93fc836fe74c9daa11c7b2848a2922 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:57.667137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30cd4d-981d-4ab8-b0ca-f258c4bda5d0 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:57.667425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30cd4d-981d-4ab8-b0ca-f258c4bda5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:57.667694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc30cd4d-981d-4ab8-b0ca-f258c4bda5d0 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:57.673089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 358/714] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:57.674758 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cabf3-475f-46b1-84b2-52a5c9b90187 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:57.675039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cabf3-475f-46b1-84b2-52a5c9b90187 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:57.675307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919cabf3-475f-46b1-84b2-52a5c9b90187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:57.694827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-919cabf3-475f-46b1-84b2-52a5c9b90187 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:57.696155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-919cabf3-475f-46b1-84b2-52a5c9b90187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:57.757371 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7682d09-d2ed-4cab-a225-df8378181729 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:57.774443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:57.789778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 359/715] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 31 18:59:57.811635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4a309-87fb-43bc-a571-8ff5565f8f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:57.812492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4a309-87fb-43bc-a571-8ff5565f8f0d None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:57.812963 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4a309-87fb-43bc-a571-8ff5565f8f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:57.813436 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4a309-87fb-43bc-a571-8ff5565f8f0d None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:57.814247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4a309-87fb-43bc-a571-8ff5565f8f0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:57.815852 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd4a309-87fb-43bc-a571-8ff5565f8f0d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:57.846245 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 357/716] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 18:59:57 2022] POST /identity/v3/users => generated 254 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 18:59:58.117312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cabf3-475f-46b1-84b2-52a5c9b90187 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:58.206854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:58.239299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919cabf3-475f-46b1-84b2-52a5c9b90187 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:58.280113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7171a4-b4c8-4c5a-bfd4-d3e6a3cd09a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:58.280745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7171a4-b4c8-4c5a-bfd4-d3e6a3cd09a0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:58.281016 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7171a4-b4c8-4c5a-bfd4-d3e6a3cd09a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:58.281309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7171a4-b4c8-4c5a-bfd4-d3e6a3cd09a0 None admin] PATH_INFO: `/v3/services/network` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:58.281881 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7171a4-b4c8-4c5a-bfd4-d3e6a3cd09a0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:58.283280 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7171a4-b4c8-4c5a-bfd4-d3e6a3cd09a0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-4d7171a4-b4c8-4c5a-bfd4-d3e6a3cd09a0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:58.291964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:58.293501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:58.293501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:58.293501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:58.293501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:58.293501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:58.293501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:58.293501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 18:59:58.293501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 18:59:58.293501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 18:59:58.293501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 18:59:58.293501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:58.293501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:58.293501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:58.293501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:58.293501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:58.293501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:58.294387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:58.294387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:58.294387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:58.294387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:58.294387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:58.294387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:58.294387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:58.294387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:58.294387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:58.294387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:58.294387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 18:59:58.294387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 18:59:58.294387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 18:59:58.294387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 18:59:58.294387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 360/717] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 18:59:58 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 18:59:58.299997 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4a309-87fb-43bc-a571-8ff5565f8f0d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:58.452075 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd4a309-87fb-43bc-a571-8ff5565f8f0d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:58.499088 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eafd2e-20c2-45e2-8593-751c11f6db5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:58.499719 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eafd2e-20c2-45e2-8593-751c11f6db5d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:58.500000 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eafd2e-20c2-45e2-8593-751c11f6db5d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:58.500272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eafd2e-20c2-45e2-8593-751c11f6db5d None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:58.500899 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eafd2e-20c2-45e2-8593-751c11f6db5d None admin] RBAC: Adding query filter params (name=network) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:58.501210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eafd2e-20c2-45e2-8593-751c11f6db5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:58.502575 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eafd2e-20c2-45e2-8593-751c11f6db5d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:58.513335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 358/718] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 18:59:58 2022] GET /identity/v3/services?name=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:58.518925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7171a4-b4c8-4c5a-bfd4-d3e6a3cd09a0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:58.661714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7171a4-b4c8-4c5a-bfd4-d3e6a3cd09a0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:58.712281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57faa15-d979-4680-8400-a51f2642ed40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:58.712805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57faa15-d979-4680-8400-a51f2642ed40 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:58.713074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57faa15-d979-4680-8400-a51f2642ed40 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:58.713361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57faa15-d979-4680-8400-a51f2642ed40 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:58.713985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57faa15-d979-4680-8400-a51f2642ed40 None admin] RBAC: Adding query filter params (type=network) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 18:59:58.714278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57faa15-d979-4680-8400-a51f2642ed40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 18:59:58.716167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57faa15-d979-4680-8400-a51f2642ed40 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 18:59:58.724719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 361/719] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 18:59:58 2022] GET /identity/v3/services?type=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 18:59:59.192398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebdd48-3400-42a7-8506-b903cf71b720 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:59.192398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebdd48-3400-42a7-8506-b903cf71b720 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:59.192398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ebdd48-3400-42a7-8506-b903cf71b720 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:59.193092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 359/720] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:59.197716 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449671aa-1743-45a9-9ba2-73d2cfe1e6df None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:59.198215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449671aa-1743-45a9-9ba2-73d2cfe1e6df None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:59.198402 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449671aa-1743-45a9-9ba2-73d2cfe1e6df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:59.234759 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-449671aa-1743-45a9-9ba2-73d2cfe1e6df None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:59.235943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-449671aa-1743-45a9-9ba2-73d2cfe1e6df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:59.238031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:59.335764 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 362/721] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:59.662936 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f63e8f-8fa5-4ac8-8f6e-9dd2d6b1bfa9 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:59.663702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f63e8f-8fa5-4ac8-8f6e-9dd2d6b1bfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:59.663702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f63e8f-8fa5-4ac8-8f6e-9dd2d6b1bfa9 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:59.665028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 360/722] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 18:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 18:59:59.678653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63ca5f-8353-4601-ba15-1eb09bbe6707 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:59.678938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63ca5f-8353-4601-ba15-1eb09bbe6707 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:59.679205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad63ca5f-8353-4601-ba15-1eb09bbe6707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:59.686784 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f63e8f-8fa5-4ac8-8f6e-9dd2d6b1bfa9 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 18:59:59.718631 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ad63ca5f-8353-4601-ba15-1eb09bbe6707 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 18:59:59.719815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ad63ca5f-8353-4601-ba15-1eb09bbe6707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 18:59:59.737830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 18:59:59.851015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 363/723] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 18:59:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 18:59:59.869973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f63e8f-8fa5-4ac8-8f6e-9dd2d6b1bfa9 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 18:59:59.921992 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b0eda-86e5-4890-8328-b25de636a5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 18:59:59.922555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b0eda-86e5-4890-8328-b25de636a5da None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 18:59:59.922830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b0eda-86e5-4890-8328-b25de636a5da None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 18:59:59.923100 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5b0eda-86e5-4890-8328-b25de636a5da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 18:59:59.937056 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5b0eda-86e5-4890-8328-b25de636a5da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 18:59:59.942292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-6f5b0eda-86e5-4890-8328-b25de636a5da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:59:59.942292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 18:59:59.942292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 18:59:59.942292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 18:59:59.942292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 18:59:59.942292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 18:59:59.942292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 18:59:59.942292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 18:59:59.942292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 18:59:59.942292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 18:59:59.942292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 18:59:59.942292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 18:59:59.942292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 18:59:59.942292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 18:59:59.942292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 18:59:59.942292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 18:59:59.944409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 18:59:59.944409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 18:59:59.944409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 18:59:59.944409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 18:59:59.944409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 18:59:59.944409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 18:59:59.944409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 18:59:59.944409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 18:59:59.944409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 18:59:59.944409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 18:59:59.944409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 18:59:59.944409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 18:59:59.944409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 18:59:59.944409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 18:59:59.944409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 18:59:59.944409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 18:59:59.946467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 18:59:59.946467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 18:59:59.946467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 18:59:59.946467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 18:59:59.946467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 18:59:59.946467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 18:59:59.946467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 18:59:59.946467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 18:59:59.946467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 18:59:59.946467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 18:59:59.946467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 18:59:59.946467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 18:59:59.948053 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 361/724] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 18:59:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 18:59:59.953208 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63ca5f-8353-4601-ba15-1eb09bbe6707 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:00.062909 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:00.111941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad63ca5f-8353-4601-ba15-1eb09bbe6707 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:00.157898 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc13bc1d-be4a-4147-938d-a48bf95283b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:00.158505 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13bc1d-be4a-4147-938d-a48bf95283b7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:00.158783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13bc1d-be4a-4147-938d-a48bf95283b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:00.159058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc13bc1d-be4a-4147-938d-a48bf95283b7 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:00.159714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13bc1d-be4a-4147-938d-a48bf95283b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:00.160015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13bc1d-be4a-4147-938d-a48bf95283b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:00.161398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc13bc1d-be4a-4147-938d-a48bf95283b7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:00.172993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 364/725] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 18:59:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:00.178471 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b0eda-86e5-4890-8328-b25de636a5da None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:00.285483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:00.290210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb9b7d-6243-459d-8845-26a4587d5d7f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:00.291069 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb9b7d-6243-459d-8845-26a4587d5d7f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:00.291598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcb9b7d-6243-459d-8845-26a4587d5d7f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:00.294030 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 365/726] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:00.298581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb5fba-dcca-43e5-baba-1fffd7fb0c3b None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:00.299104 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb5fba-dcca-43e5-baba-1fffd7fb0c3b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:00.299378 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fb5fba-dcca-43e5-baba-1fffd7fb0c3b None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:00.300722 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 366/727] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:00.301660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e16cc-bfec-43ed-b86a-7df1439aaf1c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:00.301940 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e16cc-bfec-43ed-b86a-7df1439aaf1c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:00.302213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e16cc-bfec-43ed-b86a-7df1439aaf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:00.303591 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5b0eda-86e5-4890-8328-b25de636a5da None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:00.322150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c33e16cc-bfec-43ed-b86a-7df1439aaf1c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:00.323346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c33e16cc-bfec-43ed-b86a-7df1439aaf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:00.340425 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6690df-3444-440e-8c42-19b9a76aaa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:00.341051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6690df-3444-440e-8c42-19b9a76aaa42 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:00.341364 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6690df-3444-440e-8c42-19b9a76aaa42 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:00.341644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6690df-3444-440e-8c42-19b9a76aaa42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:00.354831 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6690df-3444-440e-8c42-19b9a76aaa42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:00.356906 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6690df-3444-440e-8c42-19b9a76aaa42 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:00.371063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-aa6690df-3444-440e-8c42-19b9a76aaa42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:00.371063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:00.371063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:00.371063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:00.371063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:00.371063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:00.371063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:00.371063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:00.371063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:00.371063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:00.371063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:00.371063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:00.371063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:00.371063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:00.371063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:00.371063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:00.372041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:00.372922 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:00.372922 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:00.372922 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:00.372922 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 362/728] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:00.372922 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6690df-3444-440e-8c42-19b9a76aaa42 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:00.401229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 367/729] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:00:00.403050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a07e77-93d7-4818-8b82-bcdd23fa14d2 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:00.403327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a07e77-93d7-4818-8b82-bcdd23fa14d2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:00.403592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a07e77-93d7-4818-8b82-bcdd23fa14d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:00.421759 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-76a07e77-93d7-4818-8b82-bcdd23fa14d2 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:00.422337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-76a07e77-93d7-4818-8b82-bcdd23fa14d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:00.492866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:00.509741 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6690df-3444-440e-8c42-19b9a76aaa42 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:00.510782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 368/730] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:00:00.513610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a07e77-93d7-4818-8b82-bcdd23fa14d2 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:00.617873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3293c2e-e1e8-4180-93e8-0f1abaa0a398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:00.618483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3293c2e-e1e8-4180-93e8-0f1abaa0a398 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:00.618761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3293c2e-e1e8-4180-93e8-0f1abaa0a398 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:00.619034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3293c2e-e1e8-4180-93e8-0f1abaa0a398 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:00.619679 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3293c2e-e1e8-4180-93e8-0f1abaa0a398 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:00.621053 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3293c2e-e1e8-4180-93e8-0f1abaa0a398 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:00.630395 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a07e77-93d7-4818-8b82-bcdd23fa14d2 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:00.656896 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:00.669885 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785bd6c-8ae3-4401-907a-330e8f7a8e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:00.670896 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785bd6c-8ae3-4401-907a-330e8f7a8e35 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:00.671401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785bd6c-8ae3-4401-907a-330e8f7a8e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:00.671889 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1785bd6c-8ae3-4401-907a-330e8f7a8e35 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:00.672230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 363/731] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 19:00:00 2022] POST /identity/v3/services => generated 251 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:00:00.673675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785bd6c-8ae3-4401-907a-330e8f7a8e35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:00.674182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785bd6c-8ae3-4401-907a-330e8f7a8e35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:00.675926 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1785bd6c-8ae3-4401-907a-330e8f7a8e35 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:00.678378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3293c2e-e1e8-4180-93e8-0f1abaa0a398 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:00.689601 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 369/732] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:00.697557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785bd6c-8ae3-4401-907a-330e8f7a8e35 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:00.785713 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:00.866204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1785bd6c-8ae3-4401-907a-330e8f7a8e35 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:00.885258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3293c2e-e1e8-4180-93e8-0f1abaa0a398 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:00.909319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af616ee6-c78d-41d5-a2f4-861bf9387113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:00.910243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af616ee6-c78d-41d5-a2f4-861bf9387113 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:00.910732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af616ee6-c78d-41d5-a2f4-861bf9387113 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:00.911187 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af616ee6-c78d-41d5-a2f4-861bf9387113 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:00.919414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af616ee6-c78d-41d5-a2f4-861bf9387113 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:00.920844 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af616ee6-c78d-41d5-a2f4-861bf9387113 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:00.929527 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5687fc8-d5b5-46e4-bfb7-939d1969186a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:00.930160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5687fc8-d5b5-46e4-bfb7-939d1969186a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:00.930441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5687fc8-d5b5-46e4-bfb7-939d1969186a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:00.930715 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5687fc8-d5b5-46e4-bfb7-939d1969186a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:00.933935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-af616ee6-c78d-41d5-a2f4-861bf9387113 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:00:00.933935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:00.933935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:00.933935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:00.933935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:00.933935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:00.933935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:00.933935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:00.933935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:00.933935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:00.933935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:00.933935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:00.933935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:00:00.933935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:00:00.933935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:00:00.933935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:00.934865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:00.935792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:00.935792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:00.935792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:00:00.935792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:00:00.935792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:00:00.935792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:00:00.935792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:00:00.935792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:00:00.935792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:00:00.935792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:00.937608 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 370/733] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 19:00:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:00.938626 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af616ee6-c78d-41d5-a2f4-861bf9387113 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:00.943200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5687fc8-d5b5-46e4-bfb7-939d1969186a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:00:00.948569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-a5687fc8-d5b5-46e4-bfb7-939d1969186a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:00:00.948569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:00.948569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:00.948569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:00.948569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:00.948569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:00.948569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:00.948569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:00.948569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:00.948569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:00.948569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:00.948569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:00.948569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:00:00.948569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:00:00.948569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:00:00.948569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:00:00.950189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:00:00.950189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:00:00.950189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:00:00.950189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:00:00.950189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:00.950189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:00.950189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:00.950189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:00.950189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:00.950189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:00.950189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:00.950189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:00.950189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:00.950189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:00.950189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:00.950189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:00.951021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:00.951021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:00.951021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:00.951021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:00.951021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:00:00.951021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:00:00.951021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:00:00.951021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:00:00.951021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:00:00.951021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:00:00.951021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:00:00.951021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:00.951021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 364/734] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 19:00:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:00.951727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5687fc8-d5b5-46e4-bfb7-939d1969186a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:01.024556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:01.057736 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af616ee6-c78d-41d5-a2f4-861bf9387113 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:01.081533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5687fc8-d5b5-46e4-bfb7-939d1969186a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:01.092868 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:01.097060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009380ee-2612-4bb2-ae3b-423c74934e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:01.097987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009380ee-2612-4bb2-ae3b-423c74934e37 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:01.098534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009380ee-2612-4bb2-ae3b-423c74934e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:01.098985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009380ee-2612-4bb2-ae3b-423c74934e37 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:01.111525 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009380ee-2612-4bb2-ae3b-423c74934e37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:00:01.116916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-009380ee-2612-4bb2-ae3b-423c74934e37 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:00:01.116916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:01.116916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:01.116916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:01.116916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:01.116916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:01.116916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:01.116916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:01.116916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:01.116916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:01.116916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:01.116916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:01.116916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:00:01.116916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:00:01.116916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:00:01.116916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:00:01.117917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:00:01.117917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:00:01.117917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:00:01.117917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:00:01.117917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:01.117917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:01.117917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:01.117917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:01.117917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:01.117917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:01.117917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:01.117917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:01.117917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:01.117917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:01.117917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:01.117917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:01.118746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:01.118746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:01.118746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:01.118746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:01.118746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:00:01.118746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:00:01.118746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:00:01.118746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:00:01.118746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:00:01.118746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:00:01.118746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:00:01.118746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:01.126628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a8941-7f31-4331-9846-c1ac3cb2dd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:01.126628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a8941-7f31-4331-9846-c1ac3cb2dd68 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:01.126628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a8941-7f31-4331-9846-c1ac3cb2dd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:01.126628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a8941-7f31-4331-9846-c1ac3cb2dd68 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:01.127011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a8941-7f31-4331-9846-c1ac3cb2dd68 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:01.127011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a8941-7f31-4331-9846-c1ac3cb2dd68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:01.127011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a8941-7f31-4331-9846-c1ac3cb2dd68 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:01.127465 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 371/735] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:00:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:01.128632 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009380ee-2612-4bb2-ae3b-423c74934e37 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:01.152921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 365/736] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 19:00:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:01.153951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a8941-7f31-4331-9846-c1ac3cb2dd68 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:01.248343 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009380ee-2612-4bb2-ae3b-423c74934e37 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:01.280624 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a8941-7f31-4331-9846-c1ac3cb2dd68 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:01.291787 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a87ecc-2c03-4db3-9cb4-1c929ff28b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:01.294008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a87ecc-2c03-4db3-9cb4-1c929ff28b55 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:01.294754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a87ecc-2c03-4db3-9cb4-1c929ff28b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:01.295221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a87ecc-2c03-4db3-9cb4-1c929ff28b55 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:01.296361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a87ecc-2c03-4db3-9cb4-1c929ff28b55 None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:01.297330 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:01.297517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a87ecc-2c03-4db3-9cb4-1c929ff28b55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:01.299029 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a87ecc-2c03-4db3-9cb4-1c929ff28b55 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:01.314002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 372/737] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 19:00:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:01.314002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a87ecc-2c03-4db3-9cb4-1c929ff28b55 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:01.334685 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d5784-e3c2-4a5c-af5b-ad39ec0d35d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:01.335516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d5784-e3c2-4a5c-af5b-ad39ec0d35d4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:01.336441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d5784-e3c2-4a5c-af5b-ad39ec0d35d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:01.336876 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229d5784-e3c2-4a5c-af5b-ad39ec0d35d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:01.338002 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d5784-e3c2-4a5c-af5b-ad39ec0d35d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:01.338507 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d5784-e3c2-4a5c-af5b-ad39ec0d35d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:01.341119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229d5784-e3c2-4a5c-af5b-ad39ec0d35d4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:01.346635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:01.359405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 366/738] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:00:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:01.359405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d5784-e3c2-4a5c-af5b-ad39ec0d35d4 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:01.440773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a87ecc-2c03-4db3-9cb4-1c929ff28b55 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:01.465578 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229d5784-e3c2-4a5c-af5b-ad39ec0d35d4 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:01.477157 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306541fd-032c-4dcf-b76b-a9078d0e864a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:01.477804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306541fd-032c-4dcf-b76b-a9078d0e864a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:01.478089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306541fd-032c-4dcf-b76b-a9078d0e864a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:01.478368 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306541fd-032c-4dcf-b76b-a9078d0e864a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:01.490381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306541fd-032c-4dcf-b76b-a9078d0e864a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:01.491807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306541fd-032c-4dcf-b76b-a9078d0e864a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:01.501802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26542a84-507e-4602-99e8-987c094edc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:01.502589 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26542a84-507e-4602-99e8-987c094edc0c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:01.503038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26542a84-507e-4602-99e8-987c094edc0c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:01.503458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26542a84-507e-4602-99e8-987c094edc0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:01.506031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-306541fd-032c-4dcf-b76b-a9078d0e864a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:01.506031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:01.506031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:01.506031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:01.506031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:01.506031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:01.506031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:01.506031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:01.506031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:01.506031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:01.506031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:01.506031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:01.506031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:01.506031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:01.506031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:01.506031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:01.507148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:01.508043 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:01.508043 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:01.508043 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:01.508043 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 373/739] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:01.508043 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306541fd-032c-4dcf-b76b-a9078d0e864a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:01.520788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26542a84-507e-4602-99e8-987c094edc0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:01.522345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26542a84-507e-4602-99e8-987c094edc0c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:01.537474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-26542a84-507e-4602-99e8-987c094edc0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:01.537474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:01.537474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:01.537474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:01.537474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:01.537474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:01.537474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:01.537474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:01.537474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:01.537474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:01.537474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:01.537474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:01.537474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:01.537474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:01.537474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:01.537474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:01.538470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:01.539376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:01.539376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:01.539376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:01.539376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 367/740] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:01.540242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d72b2e-30c3-4b00-877f-19750a6b780d None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:01.540611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d72b2e-30c3-4b00-877f-19750a6b780d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:01.540953 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d72b2e-30c3-4b00-877f-19750a6b780d None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:01.542426 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 368/741] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:01.547491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d72b2e-30c3-4b00-877f-19750a6b780d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:01.579168 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:01.608934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:01.626094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306541fd-032c-4dcf-b76b-a9078d0e864a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:01.659942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d72b2e-30c3-4b00-877f-19750a6b780d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:01.661049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c39f4-f630-4855-8912-8ec862625c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:01.661711 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c39f4-f630-4855-8912-8ec862625c03 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:01.661987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c39f4-f630-4855-8912-8ec862625c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:01.662256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c39f4-f630-4855-8912-8ec862625c03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:01.674068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c39f4-f630-4855-8912-8ec862625c03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:01.675503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c39f4-f630-4855-8912-8ec862625c03 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:01.690918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-134c39f4-f630-4855-8912-8ec862625c03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:01.690918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:01.690918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:01.690918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:01.690918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:01.690918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:01.690918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:01.690918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:01.690918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:01.690918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:01.690918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:01.690918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:01.690918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:01.690918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:01.690918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:01.690918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:01.691883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:01.692778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:01.692778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:01.692778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:01.692778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 374/742] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:01.692778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c39f4-f630-4855-8912-8ec862625c03 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:01.700679 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652bb7e-3003-4527-9255-8f8fe00f771a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:01.701336 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652bb7e-3003-4527-9255-8f8fe00f771a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:01.701616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652bb7e-3003-4527-9255-8f8fe00f771a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:01.701897 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652bb7e-3003-4527-9255-8f8fe00f771a None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:01.702539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652bb7e-3003-4527-9255-8f8fe00f771a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:01.702838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652bb7e-3003-4527-9255-8f8fe00f771a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:01.704203 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652bb7e-3003-4527-9255-8f8fe00f771a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:01.717049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 369/743] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:01.719004 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116ffb5-3928-4b38-bf23-0f3573e98f95 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:01.719836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116ffb5-3928-4b38-bf23-0f3573e98f95 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:01.720464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5116ffb5-3928-4b38-bf23-0f3573e98f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:01.739850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-5116ffb5-3928-4b38-bf23-0f3573e98f95 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:01.741125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5116ffb5-3928-4b38-bf23-0f3573e98f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:01.805614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c39f4-f630-4855-8912-8ec862625c03 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:01.811430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 370/744] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 31 19:00:01.812516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116ffb5-3928-4b38-bf23-0f3573e98f95 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:01.838391 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:01.848609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45836f2-27cf-4331-877a-1fd56b7baaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:01.850209 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45836f2-27cf-4331-877a-1fd56b7baaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:01.850700 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45836f2-27cf-4331-877a-1fd56b7baaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:01.851331 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45836f2-27cf-4331-877a-1fd56b7baaa7 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:01.852557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45836f2-27cf-4331-877a-1fd56b7baaa7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:01.853332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45836f2-27cf-4331-877a-1fd56b7baaa7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:01.854913 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45836f2-27cf-4331-877a-1fd56b7baaa7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:01.859019 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:01.873876 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 375/745] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:01.875210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45836f2-27cf-4331-877a-1fd56b7baaa7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:01.940312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5116ffb5-3928-4b38-bf23-0f3573e98f95 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:02.013146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45836f2-27cf-4331-877a-1fd56b7baaa7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:02.029212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e051-71b6-4e04-ae48-8b552931c772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:02.029212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e051-71b6-4e04-ae48-8b552931c772 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:02.029212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e051-71b6-4e04-ae48-8b552931c772 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:02.029212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8e051-71b6-4e04-ae48-8b552931c772 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:02.029599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e051-71b6-4e04-ae48-8b552931c772 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:02.029599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e051-71b6-4e04-ae48-8b552931c772 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:02.031623 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8e051-71b6-4e04-ae48-8b552931c772 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:02.057502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fc4bf-8cba-4f92-a526-876f421ed1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:02.058275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fc4bf-8cba-4f92-a526-876f421ed1e7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:02.058683 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fc4bf-8cba-4f92-a526-876f421ed1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:02.059050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5fc4bf-8cba-4f92-a526-876f421ed1e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:02.066496 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5fc4bf-8cba-4f92-a526-876f421ed1e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:02.066904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 371/746] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:02.068002 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e051-71b6-4e04-ae48-8b552931c772 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:02.068341 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5fc4bf-8cba-4f92-a526-876f421ed1e7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:02.083372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-8f5fc4bf-8cba-4f92-a526-876f421ed1e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:02.083372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:02.083372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:02.083372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:02.083372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:02.083372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:02.083372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:02.083372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:02.083372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:02.083372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:02.083372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:02.083372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:02.083372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:00:02.083372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:00:02.083372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:02.084474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:02.085598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:00:02.085598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:00:02.085598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:00:02.085598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:00:02.085598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:00:02.085598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:00:02.085598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:02.085598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:02.087826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 376/747] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:02.088750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fc4bf-8cba-4f92-a526-876f421ed1e7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:02.153425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:02.179029 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:02.209531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8e051-71b6-4e04-ae48-8b552931c772 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:02.224295 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5fc4bf-8cba-4f92-a526-876f421ed1e7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:02.262870 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57700e-7611-4262-b31e-13217251d097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:02.264035 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57700e-7611-4262-b31e-13217251d097 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:02.264980 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57700e-7611-4262-b31e-13217251d097 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:02.265536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57700e-7611-4262-b31e-13217251d097 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:02.266671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86eb62-bcbe-4274-97b3-c69de459991e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:02.267319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86eb62-bcbe-4274-97b3-c69de459991e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:02.267638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86eb62-bcbe-4274-97b3-c69de459991e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:02.267945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86eb62-bcbe-4274-97b3-c69de459991e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:02.276499 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57700e-7611-4262-b31e-13217251d097 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:02.277807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86eb62-bcbe-4274-97b3-c69de459991e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:02.278205 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57700e-7611-4262-b31e-13217251d097 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:02.279988 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86eb62-bcbe-4274-97b3-c69de459991e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:02.289642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-cc57700e-7611-4262-b31e-13217251d097 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:00:02.289642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:02.289642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:02.289642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:02.289642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:02.289642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:02.289642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:02.289642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:02.289642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:02.289642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:02.289642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:02.289642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:02.289642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:00:02.289642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:00:02.289642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:00:02.289642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:02.290753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:02.291882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:02.291882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:02.291882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:00:02.291882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:00:02.291882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:00:02.291882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:00:02.291882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:00:02.291882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:00:02.291882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:00:02.291882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:02.293167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 372/748] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 19:00:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:02.298303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57700e-7611-4262-b31e-13217251d097 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:02.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-7c86eb62-bcbe-4274-97b3-c69de459991e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:00:02.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:02.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:02.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:02.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:02.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:02.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:02.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:02.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:02.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:02.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:02.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:02.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:00:02.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:00:02.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:00:02.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:02.300277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:02.301427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:02.301427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:02.301427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:00:02.301427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:00:02.301427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:00:02.301427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:00:02.301427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:00:02.301427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:00:02.301427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:00:02.301427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:02.305795 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 377/749] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:00:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:02.305795 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86eb62-bcbe-4274-97b3-c69de459991e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:02.423974 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86eb62-bcbe-4274-97b3-c69de459991e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:02.432768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57700e-7611-4262-b31e-13217251d097 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:02.441021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:02.461353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3bf39-30dc-417d-becf-54fecc9522f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:02.461956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3bf39-30dc-417d-becf-54fecc9522f8 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:02.462234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3bf39-30dc-417d-becf-54fecc9522f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:02.462499 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd3bf39-30dc-417d-becf-54fecc9522f8 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:02.463135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3bf39-30dc-417d-becf-54fecc9522f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:02.464480 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd3bf39-30dc-417d-becf-54fecc9522f8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:02.468044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab43638-9fd6-4ba0-93bb-b259bc260755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:02.468644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab43638-9fd6-4ba0-93bb-b259bc260755 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:02.468923 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab43638-9fd6-4ba0-93bb-b259bc260755 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:02.469216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab43638-9fd6-4ba0-93bb-b259bc260755 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:02.469910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab43638-9fd6-4ba0-93bb-b259bc260755 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:02.470213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab43638-9fd6-4ba0-93bb-b259bc260755 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:02.471618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab43638-9fd6-4ba0-93bb-b259bc260755 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:02.478999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:02.480044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 378/750] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 19:00:02 2022] POST /identity/v3/services => generated 240 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:00:02.490785 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 373/751] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:00:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:02.490785 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab43638-9fd6-4ba0-93bb-b259bc260755 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:02.491101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3bf39-30dc-417d-becf-54fecc9522f8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:02.627321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab43638-9fd6-4ba0-93bb-b259bc260755 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:02.640273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd3bf39-30dc-417d-becf-54fecc9522f8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:02.666871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da8e76-f8da-43eb-b870-7119caef1b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:02.667482 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da8e76-f8da-43eb-b870-7119caef1b8c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:02.668301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da8e76-f8da-43eb-b870-7119caef1b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:02.668611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da8e76-f8da-43eb-b870-7119caef1b8c None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:02.669490 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da8e76-f8da-43eb-b870-7119caef1b8c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:02.669800 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da8e76-f8da-43eb-b870-7119caef1b8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:02.671230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2da8e76-f8da-43eb-b870-7119caef1b8c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:02.680206 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eff0d6-bad2-4502-90b9-2592de9b94bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:02.681067 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eff0d6-bad2-4502-90b9-2592de9b94bf None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:02.681557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eff0d6-bad2-4502-90b9-2592de9b94bf None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:02.681899 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eff0d6-bad2-4502-90b9-2592de9b94bf None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:02.682739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eff0d6-bad2-4502-90b9-2592de9b94bf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:02.683105 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eff0d6-bad2-4502-90b9-2592de9b94bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:02.685155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eff0d6-bad2-4502-90b9-2592de9b94bf None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:02.691956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 374/752] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 31 19:00:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:02.692809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da8e76-f8da-43eb-b870-7119caef1b8c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:02.705602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 379/753] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 31 19:00:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:02.706452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eff0d6-bad2-4502-90b9-2592de9b94bf None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:02.740704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:02.758676 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:02.805554 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da8e76-f8da-43eb-b870-7119caef1b8c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:02.825116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eff0d6-bad2-4502-90b9-2592de9b94bf None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:02.846467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4585d94f-6843-4f9a-83ae-533223f87984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:02.847128 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585d94f-6843-4f9a-83ae-533223f87984 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:02.847398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585d94f-6843-4f9a-83ae-533223f87984 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:02.847665 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4585d94f-6843-4f9a-83ae-533223f87984 None admin] PATH_INFO: `/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/673a72ac9a9847e3812007b66ac7a5ea/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:02.849582 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4585d94f-6843-4f9a-83ae-533223f87984 None admin] RBAC: Authorizing `identity:create_grant(project_id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user_id=673a72ac9a9847e3812007b66ac7a5ea, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:02.850993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4585d94f-6843-4f9a-83ae-533223f87984 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:02.863315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caebcd9-064a-4f3e-9fde-46b65e93abf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:02.863927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caebcd9-064a-4f3e-9fde-46b65e93abf3 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:02.864205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caebcd9-064a-4f3e-9fde-46b65e93abf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:02.864475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caebcd9-064a-4f3e-9fde-46b65e93abf3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:02.865232 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 375/754] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 19:00:02 2022] PUT /identity/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/673a72ac9a9847e3812007b66ac7a5ea/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:00:02.866150 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaece3e-277c-4cac-b3df-42c1f28f9cfd None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:02.866431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaece3e-277c-4cac-b3df-42c1f28f9cfd None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:02.866699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaece3e-277c-4cac-b3df-42c1f28f9cfd None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:02.868083 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 376/755] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:02.873451 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaece3e-277c-4cac-b3df-42c1f28f9cfd None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:02.969748 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caebcd9-064a-4f3e-9fde-46b65e93abf3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:02.971638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caebcd9-064a-4f3e-9fde-46b65e93abf3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:02.994122 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaece3e-277c-4cac-b3df-42c1f28f9cfd None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:03.006911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:03.015809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:03.039060 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846c5fd-cf37-481d-9c6b-e7f55870bef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:03.039702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846c5fd-cf37-481d-9c6b-e7f55870bef5 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:03.039977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846c5fd-cf37-481d-9c6b-e7f55870bef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:03.040242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7846c5fd-cf37-481d-9c6b-e7f55870bef5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:03.101670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 380/756] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:00:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:00:03.103478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f376c-3171-4e04-b2f8-f70692d0069a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:03.103936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f376c-3171-4e04-b2f8-f70692d0069a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:03.104334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116f376c-3171-4e04-b2f8-f70692d0069a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:03.115980 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846c5fd-cf37-481d-9c6b-e7f55870bef5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:03.117587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7846c5fd-cf37-481d-9c6b-e7f55870bef5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:03.126279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-116f376c-3171-4e04-b2f8-f70692d0069a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:03.127867 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-116f376c-3171-4e04-b2f8-f70692d0069a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:03.227791 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 377/758] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:00:02 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:00:03.228693 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846c5fd-cf37-481d-9c6b-e7f55870bef5 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:03.233640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 381/758] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:03.235401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f376c-3171-4e04-b2f8-f70692d0069a None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:03.263735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:03.339617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:03.401906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116f376c-3171-4e04-b2f8-f70692d0069a None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:03.413300 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7846c5fd-cf37-481d-9c6b-e7f55870bef5 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:03.449101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25638143-d277-4142-bba2-614a8de49b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:03.452757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25638143-d277-4142-bba2-614a8de49b1f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:03.453125 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25638143-d277-4142-bba2-614a8de49b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:03.453627 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25638143-d277-4142-bba2-614a8de49b1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:03.472827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25638143-d277-4142-bba2-614a8de49b1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:03.475470 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25638143-d277-4142-bba2-614a8de49b1f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:03.482380 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0546a-8383-4eb2-b823-cb0812d77830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:03.483062 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0546a-8383-4eb2-b823-cb0812d77830 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:03.483665 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0546a-8383-4eb2-b823-cb0812d77830 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:03.487090 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b0546a-8383-4eb2-b823-cb0812d77830 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:03.505245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-25638143-d277-4142-bba2-614a8de49b1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:03.505245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:03.505245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:03.505245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:03.505245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:03.505245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:03.505245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:03.505245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:03.505245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:03.505245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:03.505245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:03.505245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:03.505245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:03.505245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:03.505245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:03.505245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:03.506268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:03.507318 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:03.507318 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:03.507318 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:03.509421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0546a-8383-4eb2-b823-cb0812d77830 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:03.510981 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b0546a-8383-4eb2-b823-cb0812d77830 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:03.512313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 382/759] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:03.517854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25638143-d277-4142-bba2-614a8de49b1f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:03.543441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-e9b0546a-8383-4eb2-b823-cb0812d77830 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:03.543441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:03.543441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:03.543441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:03.543441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:03.543441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:03.543441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:03.543441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:03.543441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:03.543441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:03.543441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:03.543441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:03.543441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:03.543441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:03.543441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:03.543441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:03.544503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:03.545400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:03.545400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:03.545400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:03.545400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 378/760] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:03.552979 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0546a-8383-4eb2-b823-cb0812d77830 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:03.639777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:03.655996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25638143-d277-4142-bba2-614a8de49b1f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:03.672321 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:03.677365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b0546a-8383-4eb2-b823-cb0812d77830 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:03.696693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc9a72-a798-43e2-9bc8-b8d0c880a084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:03.697622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc9a72-a798-43e2-9bc8-b8d0c880a084 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:03.698174 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc9a72-a798-43e2-9bc8-b8d0c880a084 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:03.698649 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc9a72-a798-43e2-9bc8-b8d0c880a084 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:03.699694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc9a72-a798-43e2-9bc8-b8d0c880a084 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:03.700214 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc9a72-a798-43e2-9bc8-b8d0c880a084 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:03.702001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc9a72-a798-43e2-9bc8-b8d0c880a084 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:03.712019 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c41995-ab20-44da-b0d7-468e8bf9ddcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:03.712620 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c41995-ab20-44da-b0d7-468e8bf9ddcf None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:03.712902 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c41995-ab20-44da-b0d7-468e8bf9ddcf None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:03.713174 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c41995-ab20-44da-b0d7-468e8bf9ddcf None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:03.713850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c41995-ab20-44da-b0d7-468e8bf9ddcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:03.714161 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c41995-ab20-44da-b0d7-468e8bf9ddcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:03.716034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c41995-ab20-44da-b0d7-468e8bf9ddcf None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:03.717408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 383/761] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:03.718553 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc9a72-a798-43e2-9bc8-b8d0c880a084 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:03.728305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 379/762] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:03.729137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c41995-ab20-44da-b0d7-468e8bf9ddcf None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:03.825303 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc9a72-a798-43e2-9bc8-b8d0c880a084 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:03.829492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c41995-ab20-44da-b0d7-468e8bf9ddcf None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:03.858940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:03.866080 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d5897-6620-4655-87cd-85ee21c5d629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:03.866977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d5897-6620-4655-87cd-85ee21c5d629 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:03.867087 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6b0f6-59ba-4733-b0ed-88323f6321d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:03.867593 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d5897-6620-4655-87cd-85ee21c5d629 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:03.867712 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6b0f6-59ba-4733-b0ed-88323f6321d0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:03.867960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6b0f6-59ba-4733-b0ed-88323f6321d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:03.868233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d6b0f6-59ba-4733-b0ed-88323f6321d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:03.868583 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440d5897-6620-4655-87cd-85ee21c5d629 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:03.869375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d5897-6620-4655-87cd-85ee21c5d629 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:03.871238 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440d5897-6620-4655-87cd-85ee21c5d629 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:03.875725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6b0f6-59ba-4733-b0ed-88323f6321d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:03.877290 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d6b0f6-59ba-4733-b0ed-88323f6321d0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-440d5897-6620-4655-87cd-85ee21c5d629 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:03.881447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:03.882474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:03.882474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:03.882474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:03.882474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:03.882474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:03.882474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:03.882474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:03.882474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:03.882474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:03.882474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:03.882474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:03.882474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:03.882474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:03.882474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:03.882474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:03.882474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:03.883324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:03.883324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:03.883324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:03.883324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:03.883324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:03.883324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:03.883324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:03.883324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:03.883324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:03.883324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:03.883324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:03.883324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:03.883324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:00:03.883324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:03.890709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-87d6b0f6-59ba-4733-b0ed-88323f6321d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:03.890709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:03.890709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:03.890709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:03.890709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:03.890709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:03.890709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:03.890709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:03.890709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:03.890709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:03.890709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:03.890709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:03.890709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:00:03.890709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:00:03.890709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:03.891587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:03.892494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:00:03.892494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:00:03.892494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:00:03.892494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:00:03.892494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:00:03.892494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:00:03.892494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:03.892494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:03.892494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 380/764] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:03.892494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6b0f6-59ba-4733-b0ed-88323f6321d0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:03.893664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 384/764] 10.222.0.65 () {56 vars in 1129 bytes} [Wed Aug 31 19:00:03 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:03.894861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d5897-6620-4655-87cd-85ee21c5d629 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:03.957566 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:04.014692 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d6b0f6-59ba-4733-b0ed-88323f6321d0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:04.024397 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440d5897-6620-4655-87cd-85ee21c5d629 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:04.052315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c9264-9d7d-43c1-a672-2865c650ac81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:04.052944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c9264-9d7d-43c1-a672-2865c650ac81 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:04.053247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c9264-9d7d-43c1-a672-2865c650ac81 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:04.053525 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c9264-9d7d-43c1-a672-2865c650ac81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:04.060589 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c9264-9d7d-43c1-a672-2865c650ac81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:04.062474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9c9264-9d7d-43c1-a672-2865c650ac81 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:04.071487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6f14a-5bd2-4af3-9380-0ff0ccfb60bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:04.072375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6f14a-5bd2-4af3-9380-0ff0ccfb60bf None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:04.072968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6f14a-5bd2-4af3-9380-0ff0ccfb60bf None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:04.073558 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6f14a-5bd2-4af3-9380-0ff0ccfb60bf None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:04.073760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-1c9c9264-9d7d-43c1-a672-2865c650ac81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:04.073760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:04.073760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:04.073760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:04.073760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:04.073760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:04.073760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:04.073760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:04.073760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:04.073760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:04.073760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:04.073760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:04.073760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:00:04.073760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:00:04.073760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:04.075157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:04.076111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:00:04.076111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:00:04.076111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:00:04.076111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:00:04.076111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:00:04.076111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:00:04.076111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:04.076111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:04.076111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 381/765] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:04.076615 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c9264-9d7d-43c1-a672-2865c650ac81 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:04.077838 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6f14a-5bd2-4af3-9380-0ff0ccfb60bf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:04.078354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6f14a-5bd2-4af3-9380-0ff0ccfb60bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:04.081266 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6f14a-5bd2-4af3-9380-0ff0ccfb60bf None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:04.092528 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 385/766] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 31 19:00:03 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:04.092528 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6f14a-5bd2-4af3-9380-0ff0ccfb60bf None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:04.151949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:04.205475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:04.208863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6f14a-5bd2-4af3-9380-0ff0ccfb60bf None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:04.226895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c9264-9d7d-43c1-a672-2865c650ac81 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:04.247371 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62306f-2d78-4e15-a9b0-11c86f366412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:04.248600 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62306f-2d78-4e15-a9b0-11c86f366412 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:04.249525 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62306f-2d78-4e15-a9b0-11c86f366412 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:04.249981 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62306f-2d78-4e15-a9b0-11c86f366412 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:04.251401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62306f-2d78-4e15-a9b0-11c86f366412 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:04.251889 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62306f-2d78-4e15-a9b0-11c86f366412 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:04.254316 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62306f-2d78-4e15-a9b0-11c86f366412 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:04.260925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a882a0f-f61e-4d19-a365-69cfc12e2f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:04.262155 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a882a0f-f61e-4d19-a365-69cfc12e2f02 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:04.262512 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a882a0f-f61e-4d19-a365-69cfc12e2f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:04.262857 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a882a0f-f61e-4d19-a365-69cfc12e2f02 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:04.263621 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a882a0f-f61e-4d19-a365-69cfc12e2f02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:04.263994 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a882a0f-f61e-4d19-a365-69cfc12e2f02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:04.265501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a882a0f-f61e-4d19-a365-69cfc12e2f02 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:04.271025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 386/767] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:00:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:04.272122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62306f-2d78-4e15-a9b0-11c86f366412 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:04.279018 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 382/768] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:00:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:04.279934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a882a0f-f61e-4d19-a365-69cfc12e2f02 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:04.384864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a882a0f-f61e-4d19-a365-69cfc12e2f02 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:04.407824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62306f-2d78-4e15-a9b0-11c86f366412 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:04.413564 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:04.427837 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0cbca-bd86-43e5-b334-0e187290d9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:04.428852 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0cbca-bd86-43e5-b334-0e187290d9eb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:04.429260 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0cbca-bd86-43e5-b334-0e187290d9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:04.429651 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af0cbca-bd86-43e5-b334-0e187290d9eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:04.430780 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0cbca-bd86-43e5-b334-0e187290d9eb None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=586715d11b924e2cb6f44030efd77619) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:04.431580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0cbca-bd86-43e5-b334-0e187290d9eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:04.433098 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af0cbca-bd86-43e5-b334-0e187290d9eb None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:04.440784 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 383/769] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:00:04 2022] GET /identity/v3/role_assignments?user.id=586715d11b924e2cb6f44030efd77619&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=3580c894dd974d35bc95848b8ba22598 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:04.441989 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0cbca-bd86-43e5-b334-0e187290d9eb None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:04.452357 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fda22-b392-4cd7-b5e9-707e4980a92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:04.453306 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fda22-b392-4cd7-b5e9-707e4980a92e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:04.454262 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fda22-b392-4cd7-b5e9-707e4980a92e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:04.454684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30fda22-b392-4cd7-b5e9-707e4980a92e None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:04.455481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30fda22-b392-4cd7-b5e9-707e4980a92e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:04.455928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30fda22-b392-4cd7-b5e9-707e4980a92e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:04.458194 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30fda22-b392-4cd7-b5e9-707e4980a92e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:04.466581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 387/770] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 31 19:00:04 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:04.472349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fda22-b392-4cd7-b5e9-707e4980a92e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:04.535125 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:04.609427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af0cbca-bd86-43e5-b334-0e187290d9eb None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:04.634641 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30fda22-b392-4cd7-b5e9-707e4980a92e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:04.644784 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d95de5-b47c-4a7f-a236-5548304ce02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:04.645431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d95de5-b47c-4a7f-a236-5548304ce02b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:04.645713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d95de5-b47c-4a7f-a236-5548304ce02b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:04.645986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d95de5-b47c-4a7f-a236-5548304ce02b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:04.646707 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d95de5-b47c-4a7f-a236-5548304ce02b None admin] RBAC: Adding query filter params (role.id=6e93fc836fe74c9daa11c7b2848a2922, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=dd730375a996433d982b26c8339ec4ee) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:04.647015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d95de5-b47c-4a7f-a236-5548304ce02b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:04.648438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d95de5-b47c-4a7f-a236-5548304ce02b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:04.654451 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 384/771] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:00:04 2022] GET /identity/v3/role_assignments?user.id=dd730375a996433d982b26c8339ec4ee&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=6e93fc836fe74c9daa11c7b2848a2922 => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:04.655690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452155aa-0a9e-4cd4-b764-4180c465b6d6 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:04.655974 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452155aa-0a9e-4cd4-b764-4180c465b6d6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:04.656247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452155aa-0a9e-4cd4-b764-4180c465b6d6 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:04.657786 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 385/772] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:04.670716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4a81f-9a92-4681-ac0b-d4d215166896 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:04.671173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4a81f-9a92-4681-ac0b-d4d215166896 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:04.671524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4a81f-9a92-4681-ac0b-d4d215166896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:04.677691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65827ca5-af39-47a0-8586-3017d9982345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:04.678412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65827ca5-af39-47a0-8586-3017d9982345 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:04.678692 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65827ca5-af39-47a0-8586-3017d9982345 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:04.678959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65827ca5-af39-47a0-8586-3017d9982345 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:04.679685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65827ca5-af39-47a0-8586-3017d9982345 None admin] RBAC: Adding query filter params (interface=public, service_id=a3b6ebb3d850400896531d6cc52c20f4, region_id=RegionOne) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:04.679983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65827ca5-af39-47a0-8586-3017d9982345 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:04.681425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65827ca5-af39-47a0-8586-3017d9982345 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:04.691330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 388/773] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 31 19:00:04 2022] GET /identity/v3/endpoints?service_id=a3b6ebb3d850400896531d6cc52c20f4&interface=public®ion_id=RegionOne => generated 194 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:04.702521 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65827ca5-af39-47a0-8586-3017d9982345 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:04.707352 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d5d4a81f-9a92-4681-ac0b-d4d215166896 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:04.708595 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d5d4a81f-9a92-4681-ac0b-d4d215166896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:04.762129 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:04.825641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 386/774] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:04.872470 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:04.900122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65827ca5-af39-47a0-8586-3017d9982345 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:04.954968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72672f-3173-4d43-bdd8-d263ac1b1406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:04.955958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72672f-3173-4d43-bdd8-d263ac1b1406 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:04.956442 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72672f-3173-4d43-bdd8-d263ac1b1406 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:04.956915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72672f-3173-4d43-bdd8-d263ac1b1406 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:04.957757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72672f-3173-4d43-bdd8-d263ac1b1406 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:04.959373 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72672f-3173-4d43-bdd8-d263ac1b1406 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:04.969220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 389/775] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 19:00:04 2022] GET /identity/v3/services => generated 772 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:05.120532 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7efbf-a9c3-46f6-94c2-4bdc8781003d None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:05.122059 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72672f-3173-4d43-bdd8-d263ac1b1406 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:05.122651 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7efbf-a9c3-46f6-94c2-4bdc8781003d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:05.123138 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc7efbf-a9c3-46f6-94c2-4bdc8781003d None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:05.125499 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 387/776] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:05.130643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cb615-cab6-4d85-9681-b5a1b96e017e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:05.131210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cb615-cab6-4d85-9681-b5a1b96e017e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:05.131768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6cb615-cab6-4d85-9681-b5a1b96e017e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:05.153278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4f6cb615-cab6-4d85-9681-b5a1b96e017e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:05.154388 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4f6cb615-cab6-4d85-9681-b5a1b96e017e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:05.261424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72672f-3173-4d43-bdd8-d263ac1b1406 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:05.265210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 388/777] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:05.312684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6a5e5-6038-4c82-bd07-c3030184fcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:05.313336 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6a5e5-6038-4c82-bd07-c3030184fcb2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:05.313609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6a5e5-6038-4c82-bd07-c3030184fcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:05.313926 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe6a5e5-6038-4c82-bd07-c3030184fcb2 None admin] PATH_INFO: `/v3/services/network` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:05.314504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6a5e5-6038-4c82-bd07-c3030184fcb2 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:05.315899 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe6a5e5-6038-4c82-bd07-c3030184fcb2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-5fe6a5e5-6038-4c82-bd07-c3030184fcb2 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:05.327210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:05.328221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:05.328221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:05.328221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:05.328221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:05.328221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:05.328221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:05.328221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:05.328221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:05.328221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:05.328221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:05.328221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:05.328221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:05.328221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:05.328221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:05.328221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:05.328221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:05.329153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:05.329153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:05.329153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:05.329153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:05.329153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:05.329153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:05.329153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:05.329153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:05.329153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:05.329153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:05.329153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:05.329153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:05.329153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:00:05.329153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:05.333039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 390/778] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:05 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:05.334129 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cb615-cab6-4d85-9681-b5a1b96e017e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:05.552331 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:05.558393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6cb615-cab6-4d85-9681-b5a1b96e017e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:05.588600 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6a5e5-6038-4c82-bd07-c3030184fcb2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:05.606032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2754f5-a629-49c7-adf0-b6b0af219925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:05.606638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2754f5-a629-49c7-adf0-b6b0af219925 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:05.606908 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2754f5-a629-49c7-adf0-b6b0af219925 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:05.607173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2754f5-a629-49c7-adf0-b6b0af219925 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:05.607794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2754f5-a629-49c7-adf0-b6b0af219925 None admin] RBAC: Adding query filter params (name=network) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:05.608087 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2754f5-a629-49c7-adf0-b6b0af219925 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:05.609450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2754f5-a629-49c7-adf0-b6b0af219925 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:05.621045 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 389/779] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 19:00:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:05.621448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:05.627347 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2754f5-a629-49c7-adf0-b6b0af219925 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:05.723874 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe6a5e5-6038-4c82-bd07-c3030184fcb2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:05.760641 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15425c20-914e-4b53-b65a-b41eb22b1a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:05.761318 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15425c20-914e-4b53-b65a-b41eb22b1a4a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:05.762231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15425c20-914e-4b53-b65a-b41eb22b1a4a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:05.762558 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15425c20-914e-4b53-b65a-b41eb22b1a4a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:05.773874 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15425c20-914e-4b53-b65a-b41eb22b1a4a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:00:05.779446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-15425c20-914e-4b53-b65a-b41eb22b1a4a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:00:05.779446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:05.779446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:05.779446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:05.779446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:05.779446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:05.779446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:05.779446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:05.779446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:05.779446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:05.779446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:05.779446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:05.779446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:00:05.779446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:00:05.779446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:00:05.779446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:00:05.781334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:00:05.781334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:00:05.781334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:00:05.781334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:00:05.781334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:05.781334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:05.781334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:05.781334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:05.781334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:05.781334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:05.781334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:05.781334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:05.781334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:05.781334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:05.781334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:05.781334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:05.783283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:05.783283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:05.783283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:05.783283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:05.783283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:00:05.783283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:00:05.783283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:00:05.783283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:00:05.783283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:00:05.783283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:00:05.783283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:00:05.783283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:05.783283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 391/780] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:00:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:05.789572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15425c20-914e-4b53-b65a-b41eb22b1a4a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:05.879422 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2754f5-a629-49c7-adf0-b6b0af219925 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:05.921998 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ab868-e9ac-4a97-82db-665a246d37ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:05.922610 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ab868-e9ac-4a97-82db-665a246d37ce None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:05.922893 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ab868-e9ac-4a97-82db-665a246d37ce None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:05.923164 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22ab868-e9ac-4a97-82db-665a246d37ce None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:05.923805 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ab868-e9ac-4a97-82db-665a246d37ce None admin] RBAC: Adding query filter params (type=network) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:05.924103 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ab868-e9ac-4a97-82db-665a246d37ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:05.925588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22ab868-e9ac-4a97-82db-665a246d37ce None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:05.941445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 390/781] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 19:00:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:05.945544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:05.948923 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ab868-e9ac-4a97-82db-665a246d37ce None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:05.983796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15425c20-914e-4b53-b65a-b41eb22b1a4a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:06.005757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:06.056986 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11e4cf-938f-421d-a6fd-a9f26587c509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:06.057613 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11e4cf-938f-421d-a6fd-a9f26587c509 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:06.057886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11e4cf-938f-421d-a6fd-a9f26587c509 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:06.058151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef11e4cf-938f-421d-a6fd-a9f26587c509 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:06.058889 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11e4cf-938f-421d-a6fd-a9f26587c509 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:06.059183 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11e4cf-938f-421d-a6fd-a9f26587c509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:06.060522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef11e4cf-938f-421d-a6fd-a9f26587c509 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:06.075287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 392/782] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:00:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:06.082082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11e4cf-938f-421d-a6fd-a9f26587c509 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:06.132093 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22ab868-e9ac-4a97-82db-665a246d37ce None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:06.202908 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526a72f-dc00-42e6-b517-a4aeaabc0091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:06.203537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526a72f-dc00-42e6-b517-a4aeaabc0091 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:06.203819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526a72f-dc00-42e6-b517-a4aeaabc0091 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:06.204090 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2526a72f-dc00-42e6-b517-a4aeaabc0091 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:06.204794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526a72f-dc00-42e6-b517-a4aeaabc0091 None admin] RBAC: Adding query filter params (interface=public, service_id=8054e966765a4c01b80b041e83c625f2, region_id=RegionOne) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:06.205099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526a72f-dc00-42e6-b517-a4aeaabc0091 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:06.207920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2526a72f-dc00-42e6-b517-a4aeaabc0091 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:06.225494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 391/783] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 31 19:00:05 2022] GET /identity/v3/endpoints?service_id=8054e966765a4c01b80b041e83c625f2&interface=public®ion_id=RegionOne => generated 194 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:06.233081 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526a72f-dc00-42e6-b517-a4aeaabc0091 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:06.280540 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef11e4cf-938f-421d-a6fd-a9f26587c509 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:06.348501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:06.361932 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2526a72f-dc00-42e6-b517-a4aeaabc0091 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:06.382564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed00a5-784c-4371-9809-9363d475a8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:06.383122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed00a5-784c-4371-9809-9363d475a8cd None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:06.383392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed00a5-784c-4371-9809-9363d475a8cd None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:06.383656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed00a5-784c-4371-9809-9363d475a8cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:06.399994 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3c7f3-675a-4213-a5a6-88817c7c2d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:06.400874 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3c7f3-675a-4213-a5a6-88817c7c2d51 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:06.401385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3c7f3-675a-4213-a5a6-88817c7c2d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:06.401882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3c7f3-675a-4213-a5a6-88817c7c2d51 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:06.403243 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3c7f3-675a-4213-a5a6-88817c7c2d51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:06.404813 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3c7f3-675a-4213-a5a6-88817c7c2d51 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:06.406419 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:06.411739 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 392/784] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 19:00:06 2022] GET /identity/v3/services => generated 772 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:06.413375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed00a5-784c-4371-9809-9363d475a8cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:06.415149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ed00a5-784c-4371-9809-9363d475a8cd None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:06.431082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-73ed00a5-784c-4371-9809-9363d475a8cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:06.431082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:06.431082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:06.431082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:06.431082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:06.431082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:06.431082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:06.431082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:06.431082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:06.431082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:06.431082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:06.431082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:06.431082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:06.431082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:06.431082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:06.431082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:06.432150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:06.433052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:06.433052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:06.433052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:06.435408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 393/785] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:06.441632 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3c7f3-675a-4213-a5a6-88817c7c2d51 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:06.551944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3c7f3-675a-4213-a5a6-88817c7c2d51 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:06.590998 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a1d09-3b76-4e1c-940d-1d5d004a74a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:06.591977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a1d09-3b76-4e1c-940d-1d5d004a74a2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:06.592463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a1d09-3b76-4e1c-940d-1d5d004a74a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:06.592964 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a1d09-3b76-4e1c-940d-1d5d004a74a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:06.593856 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a1d09-3b76-4e1c-940d-1d5d004a74a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:06.594363 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a1d09-3b76-4e1c-940d-1d5d004a74a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:06.595931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a1d09-3b76-4e1c-940d-1d5d004a74a2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:06.608825 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 393/786] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:06.614998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed00a5-784c-4371-9809-9363d475a8cd None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:06.727339 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed00a5-784c-4371-9809-9363d475a8cd None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:06.768430 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd7788-b5c0-47b9-b95c-65b254914daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:06.769450 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd7788-b5c0-47b9-b95c-65b254914daa None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:06.769934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd7788-b5c0-47b9-b95c-65b254914daa None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:06.770428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd7788-b5c0-47b9-b95c-65b254914daa None admin] PATH_INFO: `/v3/users/nova` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:06.778729 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd7788-b5c0-47b9-b95c-65b254914daa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:06.780478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd7788-b5c0-47b9-b95c-65b254914daa None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:06.792216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-03dd7788-b5c0-47b9-b95c-65b254914daa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:00:06.792216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:06.792216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:06.792216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:06.792216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:06.792216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:06.792216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:06.792216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:06.792216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:06.792216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:06.792216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:06.792216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:06.792216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:00:06.792216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:00:06.792216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:00:06.792216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:06.793229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:06.794204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:06.794204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:06.794204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:00:06.794204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:00:06.794204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:00:06.794204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:00:06.794204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:00:06.794204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:00:06.794204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:00:06.794204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:06.796823 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 394/787] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 19:00:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:06.802678 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a1d09-3b76-4e1c-940d-1d5d004a74a2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:06.816866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caa971-16d6-484a-9656-da61dd0b6003 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:06.817499 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caa971-16d6-484a-9656-da61dd0b6003 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:06.817977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48caa971-16d6-484a-9656-da61dd0b6003 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:06.820446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 395/788] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:06.825538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:06.829956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d60743-ec67-4c50-8a88-661e22b0f110 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:06.830454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d60743-ec67-4c50-8a88-661e22b0f110 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:06.831331 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d60743-ec67-4c50-8a88-661e22b0f110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:06.852724 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-33d60743-ec67-4c50-8a88-661e22b0f110 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:06.853920 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-33d60743-ec67-4c50-8a88-661e22b0f110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:06.921660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:06.931244 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a1d09-3b76-4e1c-940d-1d5d004a74a2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:06.954950 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 396/789] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:06 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:06.968070 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97d6610-4201-4a16-bd03-4539697fd401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:06.968936 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d6610-4201-4a16-bd03-4539697fd401 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:06.969423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d6610-4201-4a16-bd03-4539697fd401 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:06.969875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97d6610-4201-4a16-bd03-4539697fd401 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:06.970792 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d6610-4201-4a16-bd03-4539697fd401 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:06.971269 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d6610-4201-4a16-bd03-4539697fd401 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:06.972969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97d6610-4201-4a16-bd03-4539697fd401 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:06.991633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 394/790] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 19:00:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:06.998149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d60743-ec67-4c50-8a88-661e22b0f110 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:07.041062 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a4ae-b00e-4e07-bcd7-256b33c8e4ea None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:07.041414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a4ae-b00e-4e07-bcd7-256b33c8e4ea None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:07.041638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2282a4ae-b00e-4e07-bcd7-256b33c8e4ea None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:07.043179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 395/791] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:00:07.048513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be5b06-531e-4c5f-a2fa-34cdbe3d5428 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:07.049046 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be5b06-531e-4c5f-a2fa-34cdbe3d5428 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:07.049576 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be5b06-531e-4c5f-a2fa-34cdbe3d5428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:07.076702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e6be5b06-531e-4c5f-a2fa-34cdbe3d5428 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:07.079021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e6be5b06-531e-4c5f-a2fa-34cdbe3d5428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:07.169560 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:07.179118 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d60743-ec67-4c50-8a88-661e22b0f110 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:07.180887 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 396/792] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:00:07.212545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53130215-544d-407e-aa14-d16935d6b2b4 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:07.212545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53130215-544d-407e-aa14-d16935d6b2b4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:07.212545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53130215-544d-407e-aa14-d16935d6b2b4 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:07.213768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 397/793] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:07.219944 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb248a-9772-4ee7-8398-7b7d29758772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:07.220353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb248a-9772-4ee7-8398-7b7d29758772 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:07.220630 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb248a-9772-4ee7-8398-7b7d29758772 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:07.220898 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bb248a-9772-4ee7-8398-7b7d29758772 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:07.222542 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd55c14-acbd-4f78-b74f-4e96422ddba9 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:07.222822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd55c14-acbd-4f78-b74f-4e96422ddba9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:07.223091 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd55c14-acbd-4f78-b74f-4e96422ddba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:07.252933 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-cbd55c14-acbd-4f78-b74f-4e96422ddba9 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:07.254160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cbd55c14-acbd-4f78-b74f-4e96422ddba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:07.315027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb248a-9772-4ee7-8398-7b7d29758772 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:07.316844 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bb248a-9772-4ee7-8398-7b7d29758772 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:07.388516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 398/794] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:07 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:07.394154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd55c14-acbd-4f78-b74f-4e96422ddba9 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:07.457278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:07.568017 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd55c14-acbd-4f78-b74f-4e96422ddba9 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:07.584229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 397/795] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:00:06 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:00:07.585406 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb248a-9772-4ee7-8398-7b7d29758772 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:07.603606 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:07.611167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbfbe9-1990-41f2-b580-f76629c2ffce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:07.612157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbfbe9-1990-41f2-b580-f76629c2ffce None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:07.612537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbfbe9-1990-41f2-b580-f76629c2ffce None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:07.612911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfbfbe9-1990-41f2-b580-f76629c2ffce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:07.625695 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfbfbe9-1990-41f2-b580-f76629c2ffce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:00:07.630669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-8bfbfbe9-1990-41f2-b580-f76629c2ffce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:00:07.630669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:07.630669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:07.630669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:07.630669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:07.630669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:07.630669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:07.630669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:07.630669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:07.630669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:07.630669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:07.630669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:07.630669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:00:07.630669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:00:07.630669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:00:07.630669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:00:07.631573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:00:07.631573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:00:07.631573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:00:07.631573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:00:07.631573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:07.631573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:07.631573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:07.631573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:07.631573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:07.631573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:07.631573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:07.631573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:07.631573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:07.631573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:07.631573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:07.631573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:07.632977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:07.632977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:07.632977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:07.632977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:07.632977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:00:07.632977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:00:07.632977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:00:07.632977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:00:07.632977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:00:07.632977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:00:07.632977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:00:07.632977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:07.632977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 399/796] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:00:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:07.632977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbfbe9-1990-41f2-b580-f76629c2ffce None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:07.700072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:07.724762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bb248a-9772-4ee7-8398-7b7d29758772 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:07.740410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfbfbe9-1990-41f2-b580-f76629c2ffce None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:07.763340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f9a66-0886-4caf-a179-130852fa3b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:07.763970 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f9a66-0886-4caf-a179-130852fa3b7a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:07.764252 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f9a66-0886-4caf-a179-130852fa3b7a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:07.764519 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3f9a66-0886-4caf-a179-130852fa3b7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:07.775172 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3f9a66-0886-4caf-a179-130852fa3b7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:00:07.777780 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8250014-4b2f-4baf-a4b2-2b7c4ebe48f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:07.778402 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8250014-4b2f-4baf-a4b2-2b7c4ebe48f0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:07.778674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8250014-4b2f-4baf-a4b2-2b7c4ebe48f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:07.778938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8250014-4b2f-4baf-a4b2-2b7c4ebe48f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:07.780418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-9f3f9a66-0886-4caf-a179-130852fa3b7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:00:07.780418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:07.780418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:07.780418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:07.780418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:07.780418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:07.780418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:07.780418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:07.780418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:07.780418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:07.780418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:07.780418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:07.780418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:00:07.780418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:00:07.780418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:00:07.780418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:00:07.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:00:07.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:00:07.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:00:07.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:00:07.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:07.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:07.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:07.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:07.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:07.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:07.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:07.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:07.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:07.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:07.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:07.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:07.782217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:07.782217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:07.782217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:07.782217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:07.782217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:00:07.782217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:00:07.782217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:00:07.782217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:00:07.782217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:00:07.782217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:00:07.782217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:00:07.782217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:07.782217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 398/797] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 19:00:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:07.782921 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f9a66-0886-4caf-a179-130852fa3b7a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:07.795990 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8250014-4b2f-4baf-a4b2-2b7c4ebe48f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:07.797450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8250014-4b2f-4baf-a4b2-2b7c4ebe48f0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:07.812662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-a8250014-4b2f-4baf-a4b2-2b7c4ebe48f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:07.812662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:07.812662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:07.812662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:07.812662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:07.812662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:07.812662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:07.812662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:07.812662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:07.812662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:07.812662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:07.812662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:07.812662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:07.812662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:07.812662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:07.812662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:07.813753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:07.814633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:07.814633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:07.814633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:07.814633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 400/798] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:07.814633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8250014-4b2f-4baf-a4b2-2b7c4ebe48f0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:07.862512 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:07.901649 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3f9a66-0886-4caf-a179-130852fa3b7a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:07.925494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8250014-4b2f-4baf-a4b2-2b7c4ebe48f0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:07.938129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a9f8e-af7d-49c5-bb91-55117bc42c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:07.938882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a9f8e-af7d-49c5-bb91-55117bc42c6d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:07.939533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a9f8e-af7d-49c5-bb91-55117bc42c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:07.939814 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a9f8e-af7d-49c5-bb91-55117bc42c6d None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:07.940477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a9f8e-af7d-49c5-bb91-55117bc42c6d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:07.940779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a9f8e-af7d-49c5-bb91-55117bc42c6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:07.942216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a9f8e-af7d-49c5-bb91-55117bc42c6d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:07.951210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 399/799] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:00:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:07.951851 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a9f8e-af7d-49c5-bb91-55117bc42c6d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:07.954460 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:07.965568 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f11c8-6a5b-4801-a140-5a1ae38329f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:07.966129 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f11c8-6a5b-4801-a140-5a1ae38329f5 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:07.966399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f11c8-6a5b-4801-a140-5a1ae38329f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:07.966764 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f11c8-6a5b-4801-a140-5a1ae38329f5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:07.967223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f11c8-6a5b-4801-a140-5a1ae38329f5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:07.968553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5f11c8-6a5b-4801-a140-5a1ae38329f5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-9e5f11c8-6a5b-4801-a140-5a1ae38329f5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:07.979313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:07.980484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:07.980484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:07.980484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:07.980484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:07.980484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:07.980484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:07.980484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:07.980484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:07.980484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:07.980484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:07.980484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:07.980484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:07.980484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:07.980484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:07.980484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:07.980484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:07.981354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:07.981354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:07.981354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:07.981354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:07.981354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:07.981354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:07.981354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:07.981354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:07.981354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:07.981354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:07.981354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:07.981354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:07.981354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 31 19:00:07.981354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:07.981354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 401/800] 10.222.0.65 () {56 vars in 1129 bytes} [Wed Aug 31 19:00:07 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:07.982197 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f11c8-6a5b-4801-a140-5a1ae38329f5 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:08.060959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a9f8e-af7d-49c5-bb91-55117bc42c6d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:08.089593 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f11c8-6a5b-4801-a140-5a1ae38329f5 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:08.095735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7391b-5117-4e1b-ad2f-471a0f5f68d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:08.096328 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7391b-5117-4e1b-ad2f-471a0f5f68d3 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:08.096601 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7391b-5117-4e1b-ad2f-471a0f5f68d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:08.096871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7391b-5117-4e1b-ad2f-471a0f5f68d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:08.097548 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7391b-5117-4e1b-ad2f-471a0f5f68d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:08.097842 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7391b-5117-4e1b-ad2f-471a0f5f68d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:08.099187 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe7391b-5117-4e1b-ad2f-471a0f5f68d3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:08.100584 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:08.107998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 400/801] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 19:00:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:08.108651 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7391b-5117-4e1b-ad2f-471a0f5f68d3 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:08.139119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a5eeb-0700-452f-b2d9-d9acbbe8ec85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:08.139638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a5eeb-0700-452f-b2d9-d9acbbe8ec85 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:08.139916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a5eeb-0700-452f-b2d9-d9acbbe8ec85 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:08.140180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a5eeb-0700-452f-b2d9-d9acbbe8ec85 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:08.140816 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a5eeb-0700-452f-b2d9-d9acbbe8ec85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:08.141112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a5eeb-0700-452f-b2d9-d9acbbe8ec85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:08.142518 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a5eeb-0700-452f-b2d9-d9acbbe8ec85 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:08.154501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 402/802] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:08.155313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a5eeb-0700-452f-b2d9-d9acbbe8ec85 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:08.181657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:08.230632 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7391b-5117-4e1b-ad2f-471a0f5f68d3 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:08.262584 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a5eeb-0700-452f-b2d9-d9acbbe8ec85 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:08.265026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f08b2d-b3f5-4311-b22b-aaa1cc3377f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:08.265681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f08b2d-b3f5-4311-b22b-aaa1cc3377f9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:08.265952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f08b2d-b3f5-4311-b22b-aaa1cc3377f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:08.266214 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f08b2d-b3f5-4311-b22b-aaa1cc3377f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:08.276273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f08b2d-b3f5-4311-b22b-aaa1cc3377f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:08.278240 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f08b2d-b3f5-4311-b22b-aaa1cc3377f9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:08.294960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-b9f08b2d-b3f5-4311-b22b-aaa1cc3377f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:08.294960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:08.294960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:08.294960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:08.294960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:08.294960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:08.294960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:08.294960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:08.294960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:08.294960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:08.294960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:08.294960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:08.294960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:08.294960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:08.294960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:08.294960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:08.295947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:08.296823 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:08.296823 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:08.296823 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:08.297000 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7a4d2-66e1-40c8-a5f2-a834bc32740c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:08.297368 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7a4d2-66e1-40c8-a5f2-a834bc32740c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:08.297643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7a4d2-66e1-40c8-a5f2-a834bc32740c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:08.297909 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a7a4d2-66e1-40c8-a5f2-a834bc32740c None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:08.298524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7a4d2-66e1-40c8-a5f2-a834bc32740c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:08.298817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7a4d2-66e1-40c8-a5f2-a834bc32740c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:08.299189 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 401/803] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:08.300082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f08b2d-b3f5-4311-b22b-aaa1cc3377f9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:08.300223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a7a4d2-66e1-40c8-a5f2-a834bc32740c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:08.307696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 403/804] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 31 19:00:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:08.309093 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7a4d2-66e1-40c8-a5f2-a834bc32740c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:08.341799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:08.410976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f08b2d-b3f5-4311-b22b-aaa1cc3377f9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:08.418185 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a7a4d2-66e1-40c8-a5f2-a834bc32740c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:08.427928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:08.448840 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf9bb2-234a-405c-a826-ae40ac668105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:08.450120 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf9bb2-234a-405c-a826-ae40ac668105 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:08.450391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf9bb2-234a-405c-a826-ae40ac668105 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:08.450660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bf9bb2-234a-405c-a826-ae40ac668105 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:08.455305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760869b-9ca3-4813-b119-abddc472fcf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:08.456093 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760869b-9ca3-4813-b119-abddc472fcf5 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:08.456535 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760869b-9ca3-4813-b119-abddc472fcf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:08.456938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c760869b-9ca3-4813-b119-abddc472fcf5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:08.461351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf9bb2-234a-405c-a826-ae40ac668105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:08.462924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bf9bb2-234a-405c-a826-ae40ac668105 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:08.464812 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760869b-9ca3-4813-b119-abddc472fcf5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:08.466246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c760869b-9ca3-4813-b119-abddc472fcf5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:08.477298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-c760869b-9ca3-4813-b119-abddc472fcf5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:08.477298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:08.477298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:08.477298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:08.477298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:08.477298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:08.477298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:08.477298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:08.477298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:08.477298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:08.477298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:08.477298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:08.477298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:00:08.477298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:00:08.477298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:08.478325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:08.479200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:00:08.479200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:00:08.479200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:00:08.479200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:00:08.479200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:00:08.479200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:00:08.479200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:08.479200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:08.479200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 404/805] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:08.479200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760869b-9ca3-4813-b119-abddc472fcf5 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:08.479989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-94bf9bb2-234a-405c-a826-ae40ac668105 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:08.479989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:08.479989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:08.479989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:08.479989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:08.479989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:08.479989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:08.479989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:08.479989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:08.479989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:08.479989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:08.479989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:08.479989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:08.479989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:08.479989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:08.479989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:08.480923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:08.486001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:08.486001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:08.486001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:08.486001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 402/806] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:08.486001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf9bb2-234a-405c-a826-ae40ac668105 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:08.577965 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:08.588433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c760869b-9ca3-4813-b119-abddc472fcf5 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:08.591954 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bf9bb2-234a-405c-a826-ae40ac668105 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:08.623200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f84abb-d9b7-4cb6-97d5-84920aeeba2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:08.623854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f84abb-d9b7-4cb6-97d5-84920aeeba2a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:08.624329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f84abb-d9b7-4cb6-97d5-84920aeeba2a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:08.624596 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f84abb-d9b7-4cb6-97d5-84920aeeba2a None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:08.625297 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f84abb-d9b7-4cb6-97d5-84920aeeba2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:08.625592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f84abb-d9b7-4cb6-97d5-84920aeeba2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:08.626927 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f84abb-d9b7-4cb6-97d5-84920aeeba2a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:08.627570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11975e16-4114-4700-8387-76ed03a85bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:08.628487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11975e16-4114-4700-8387-76ed03a85bfe None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:08.628923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11975e16-4114-4700-8387-76ed03a85bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:08.629794 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11975e16-4114-4700-8387-76ed03a85bfe None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:08.630577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11975e16-4114-4700-8387-76ed03a85bfe None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:08.631015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11975e16-4114-4700-8387-76ed03a85bfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:08.632539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11975e16-4114-4700-8387-76ed03a85bfe None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:08.643291 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 403/808] 10.222.0.65 () {56 vars in 1124 bytes} [Wed Aug 31 19:00:08 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:08.643471 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 405/808] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:08.643565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11975e16-4114-4700-8387-76ed03a85bfe None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:08.643649 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f84abb-d9b7-4cb6-97d5-84920aeeba2a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:08.675307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:08.745392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f84abb-d9b7-4cb6-97d5-84920aeeba2a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:08.747146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11975e16-4114-4700-8387-76ed03a85bfe None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:08.778842 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b4a3a-b4b6-4306-b538-447249efea4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:08.779039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ee405-7b06-4735-9091-114fb08eb5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:08.779439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b4a3a-b4b6-4306-b538-447249efea4d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:08.779624 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ee405-7b06-4735-9091-114fb08eb5ab None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:08.779720 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b4a3a-b4b6-4306-b538-447249efea4d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:08.779894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ee405-7b06-4735-9091-114fb08eb5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:08.779985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b4a3a-b4b6-4306-b538-447249efea4d None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:08.780215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ee405-7b06-4735-9091-114fb08eb5ab None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:08.780593 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b4a3a-b4b6-4306-b538-447249efea4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:08.780886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ee405-7b06-4735-9091-114fb08eb5ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:08.780975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b4a3a-b4b6-4306-b538-447249efea4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:08.781182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ee405-7b06-4735-9091-114fb08eb5ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:08.782268 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b4a3a-b4b6-4306-b538-447249efea4d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:08.782603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ee405-7b06-4735-9091-114fb08eb5ab None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:08.793805 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 406/809] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:08.794402 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b4a3a-b4b6-4306-b538-447249efea4d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:08.794914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 404/810] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:00:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:08.795758 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ee405-7b06-4735-9091-114fb08eb5ab None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:08.811333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:08.887164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:08.896449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ee405-7b06-4735-9091-114fb08eb5ab None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:08.898774 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b4a3a-b4b6-4306-b538-447249efea4d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:08.931616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28324ae3-a6d4-4a75-abee-3ae0a15b2d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:08.931803 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd6e7a-1ec9-4a0d-af48-a878527ba694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:08.932139 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28324ae3-a6d4-4a75-abee-3ae0a15b2d57 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:08.932237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd6e7a-1ec9-4a0d-af48-a878527ba694 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:08.932482 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28324ae3-a6d4-4a75-abee-3ae0a15b2d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:08.932631 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd6e7a-1ec9-4a0d-af48-a878527ba694 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:08.932790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd6e7a-1ec9-4a0d-af48-a878527ba694 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:08.932949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28324ae3-a6d4-4a75-abee-3ae0a15b2d57 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:08.934581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28324ae3-a6d4-4a75-abee-3ae0a15b2d57 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:08.935958 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28324ae3-a6d4-4a75-abee-3ae0a15b2d57 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:08.941089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd6e7a-1ec9-4a0d-af48-a878527ba694 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:08.942539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd6e7a-1ec9-4a0d-af48-a878527ba694 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:08.954642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-b5fd6e7a-1ec9-4a0d-af48-a878527ba694 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:00:08.954642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:08.954642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:08.954642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:08.954642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:08.954642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:08.954642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:08.954642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:08.954642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:08.954642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:08.954642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:08.954642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:08.954642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:00:08.954642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:00:08.954642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:00:08.954642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:08.955588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:08.956572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:08.956572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:08.956572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:00:08.956572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:00:08.956572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:00:08.956572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:00:08.956572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:00:08.956572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:00:08.956572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:00:08.956572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:08.956572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 405/811] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:00:08 2022] GET /identity/v3/users/swift => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:08.957215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6b0bb-2315-4006-8f5f-829822a55ced None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:08.957215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6b0bb-2315-4006-8f5f-829822a55ced None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:08.957676 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6b0bb-2315-4006-8f5f-829822a55ced None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:08.958779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 406/812] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:08 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:08.959295 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6b0bb-2315-4006-8f5f-829822a55ced None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:08.965554 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 407/813] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:00:08 2022] POST /identity/v3/endpoints => generated 346 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:00:08.966536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28324ae3-a6d4-4a75-abee-3ae0a15b2d57 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:09.037696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:09.086021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28324ae3-a6d4-4a75-abee-3ae0a15b2d57 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:09.096628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6b0bb-2315-4006-8f5f-829822a55ced None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:09.126295 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2d065-c595-40ae-b07b-f1b1c7d028cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:09.126903 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2d065-c595-40ae-b07b-f1b1c7d028cf None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:09.127182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2d065-c595-40ae-b07b-f1b1c7d028cf None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:09.127451 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2d065-c595-40ae-b07b-f1b1c7d028cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:09.128438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2d065-c595-40ae-b07b-f1b1c7d028cf None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=673a72ac9a9847e3812007b66ac7a5ea) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:09.128560 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2d065-c595-40ae-b07b-f1b1c7d028cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:09.130209 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2d065-c595-40ae-b07b-f1b1c7d028cf None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:09.136192 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 408/814] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:00:08 2022] GET /identity/v3/role_assignments?user.id=673a72ac9a9847e3812007b66ac7a5ea&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=3580c894dd974d35bc95848b8ba22598 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:09.136821 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2d065-c595-40ae-b07b-f1b1c7d028cf None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:09.136963 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40dd06f-05d7-48db-828f-2e18b831f9aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:09.137727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dd06f-05d7-48db-828f-2e18b831f9aa None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:09.138006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dd06f-05d7-48db-828f-2e18b831f9aa None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:09.138298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40dd06f-05d7-48db-828f-2e18b831f9aa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:09.146865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40dd06f-05d7-48db-828f-2e18b831f9aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:09.148333 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40dd06f-05d7-48db-828f-2e18b831f9aa None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:09.161998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-e40dd06f-05d7-48db-828f-2e18b831f9aa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:00:09.161998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:09.161998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:09.161998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:09.161998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:09.161998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:09.161998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:09.161998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:09.161998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:09.161998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:09.161998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:09.161998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:09.161998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:00:09.161998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:00:09.161998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:00:09.161998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:09.163058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:09.164006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:09.164006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:09.164006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:00:09.164006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:00:09.164006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:00:09.164006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:00:09.164006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:00:09.164006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:00:09.164006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:00:09.164006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:09.169214 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 407/815] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 19:00:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:09.170645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224ae1b-6ca4-41b1-9497-7e32ad8c8d43 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:09.170926 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224ae1b-6ca4-41b1-9497-7e32ad8c8d43 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:09.171195 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6224ae1b-6ca4-41b1-9497-7e32ad8c8d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:09.200673 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6224ae1b-6ca4-41b1-9497-7e32ad8c8d43 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:09.202209 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6224ae1b-6ca4-41b1-9497-7e32ad8c8d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:09.211728 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:09.278879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2d065-c595-40ae-b07b-f1b1c7d028cf None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:09.300268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 408/816] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:09.300268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224ae1b-6ca4-41b1-9497-7e32ad8c8d43 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:09.321769 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:09.337356 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a622ba5-9a13-4374-9373-316844b892b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:09.338053 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a622ba5-9a13-4374-9373-316844b892b0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:09.338401 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a622ba5-9a13-4374-9373-316844b892b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:09.338961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a622ba5-9a13-4374-9373-316844b892b0 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:09.339637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a622ba5-9a13-4374-9373-316844b892b0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:09.340624 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a622ba5-9a13-4374-9373-316844b892b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:09.342048 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a622ba5-9a13-4374-9373-316844b892b0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:09.372114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 409/817] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 31 19:00:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:09.378284 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a622ba5-9a13-4374-9373-316844b892b0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:09.424063 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6224ae1b-6ca4-41b1-9497-7e32ad8c8d43 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:09.470001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cd261-e960-494e-b2e2-48a49148ff03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:09.471210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cd261-e960-494e-b2e2-48a49148ff03 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:09.471695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cd261-e960-494e-b2e2-48a49148ff03 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:09.472187 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4cd261-e960-494e-b2e2-48a49148ff03 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:09.473709 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cd261-e960-494e-b2e2-48a49148ff03 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:09.474534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cd261-e960-494e-b2e2-48a49148ff03 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:09.477327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4cd261-e960-494e-b2e2-48a49148ff03 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:09.498071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:09.501805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 409/818] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 31 19:00:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:09.508062 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cd261-e960-494e-b2e2-48a49148ff03 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:09.539205 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a622ba5-9a13-4374-9373-316844b892b0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:09.602347 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d301b-566d-4aa1-8945-7eb7b1de17d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:09.602976 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d301b-566d-4aa1-8945-7eb7b1de17d4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:09.603259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d301b-566d-4aa1-8945-7eb7b1de17d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:09.603532 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d301b-566d-4aa1-8945-7eb7b1de17d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:09.614056 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d301b-566d-4aa1-8945-7eb7b1de17d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:09.615490 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249d301b-566d-4aa1-8945-7eb7b1de17d4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:09.632664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-249d301b-566d-4aa1-8945-7eb7b1de17d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:09.632664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:09.632664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:09.632664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:09.632664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:09.632664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:09.632664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:09.632664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:09.632664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:09.632664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:09.632664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:09.632664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:09.632664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:09.632664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:09.632664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:09.632664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:09.633688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:09.634651 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:09.634651 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:09.634651 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:09.637411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 410/819] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:09.638413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d301b-566d-4aa1-8945-7eb7b1de17d4 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:09.655143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4cd261-e960-494e-b2e2-48a49148ff03 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:09.679364 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:09.701649 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020eb9cb-ecc8-4fc7-8022-0f0f658dd36e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:09.702622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020eb9cb-ecc8-4fc7-8022-0f0f658dd36e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:09.703070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020eb9cb-ecc8-4fc7-8022-0f0f658dd36e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:09.703614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020eb9cb-ecc8-4fc7-8022-0f0f658dd36e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:09.717471 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020eb9cb-ecc8-4fc7-8022-0f0f658dd36e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:09.719427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020eb9cb-ecc8-4fc7-8022-0f0f658dd36e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:09.736025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-020eb9cb-ecc8-4fc7-8022-0f0f658dd36e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:09.736025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:09.736025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:09.736025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:09.736025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:09.736025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:09.736025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:09.736025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:09.736025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:09.736025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:09.736025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:09.736025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:09.736025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:09.736025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:09.736025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:09.736025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:09.737176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:09.738317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:09.738317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:09.738317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:09.744126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 410/820] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:09.744126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020eb9cb-ecc8-4fc7-8022-0f0f658dd36e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:09.764776 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d301b-566d-4aa1-8945-7eb7b1de17d4 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:09.813943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:09.817582 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb7742-05a3-4669-ba46-f270cbef76e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:09.818749 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb7742-05a3-4669-ba46-f270cbef76e5 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:09.819678 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb7742-05a3-4669-ba46-f270cbef76e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:09.820095 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb7742-05a3-4669-ba46-f270cbef76e5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:09.820843 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb7742-05a3-4669-ba46-f270cbef76e5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:09.822850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb7742-05a3-4669-ba46-f270cbef76e5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-f1fb7742-05a3-4669-ba46-f270cbef76e5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:09.835617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:09.836757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:09.836757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:09.836757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:09.836757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:09.836757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:09.836757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:09.836757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:09.836757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:09.836757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:09.836757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:09.836757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:09.836757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:09.836757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:09.836757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:09.836757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:09.836757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:09.837630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:09.837630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:09.837630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:09.837630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:09.837630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:09.837630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:09.837630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:09.837630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:09.837630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:09.837630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:09.837630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:09.837630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:09.837630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 31 19:00:09.837630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:09.840066 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 411/821] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:09 2022] GET /identity/v3/services/network => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:09.841024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb7742-05a3-4669-ba46-f270cbef76e5 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:09.877419 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020eb9cb-ecc8-4fc7-8022-0f0f658dd36e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:09.913565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61550a2a-9806-4a4e-a8da-876fcdcd9839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:09.914492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61550a2a-9806-4a4e-a8da-876fcdcd9839 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:09.914954 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61550a2a-9806-4a4e-a8da-876fcdcd9839 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:09.915398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61550a2a-9806-4a4e-a8da-876fcdcd9839 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:09.916198 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61550a2a-9806-4a4e-a8da-876fcdcd9839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:09.916670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61550a2a-9806-4a4e-a8da-876fcdcd9839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:09.918264 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61550a2a-9806-4a4e-a8da-876fcdcd9839 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:09.938105 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 411/822] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:09.939259 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61550a2a-9806-4a4e-a8da-876fcdcd9839 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:09.957084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:09.961343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb7742-05a3-4669-ba46-f270cbef76e5 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:10.001270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ee40e-5125-49d8-b2ec-f6b3447c5a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:10.002246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ee40e-5125-49d8-b2ec-f6b3447c5a3d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:10.002691 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ee40e-5125-49d8-b2ec-f6b3447c5a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:10.003118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44ee40e-5125-49d8-b2ec-f6b3447c5a3d None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:10.003900 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ee40e-5125-49d8-b2ec-f6b3447c5a3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:10.004358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ee40e-5125-49d8-b2ec-f6b3447c5a3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:10.006115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44ee40e-5125-49d8-b2ec-f6b3447c5a3d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:10.020273 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 412/823] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:10.021365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ee40e-5125-49d8-b2ec-f6b3447c5a3d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:10.049504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61550a2a-9806-4a4e-a8da-876fcdcd9839 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:10.085580 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:10.090021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77282db-5fa0-4d27-bae7-fd6f87a2e2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:10.090956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77282db-5fa0-4d27-bae7-fd6f87a2e2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:10.091239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77282db-5fa0-4d27-bae7-fd6f87a2e2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:10.091610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77282db-5fa0-4d27-bae7-fd6f87a2e2f0 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:10.092410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77282db-5fa0-4d27-bae7-fd6f87a2e2f0 None admin] RBAC: Adding query filter params (name=network) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:10.092750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77282db-5fa0-4d27-bae7-fd6f87a2e2f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:10.095175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77282db-5fa0-4d27-bae7-fd6f87a2e2f0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:10.104873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 412/824] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 19:00:09 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:10.104873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77282db-5fa0-4d27-bae7-fd6f87a2e2f0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:10.137920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44ee40e-5125-49d8-b2ec-f6b3447c5a3d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:10.176418 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bedad-4e22-436e-bac7-2ea6deb911d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:10.177447 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bedad-4e22-436e-bac7-2ea6deb911d9 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:10.178070 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bedad-4e22-436e-bac7-2ea6deb911d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:10.178544 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bedad-4e22-436e-bac7-2ea6deb911d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:10.186513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bedad-4e22-436e-bac7-2ea6deb911d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:10.188183 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bedad-4e22-436e-bac7-2ea6deb911d9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:10.200708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-1e1bedad-4e22-436e-bac7-2ea6deb911d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:10.200708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:10.200708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:10.200708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:10.200708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:10.200708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:10.200708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:10.200708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:10.200708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:10.200708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:10.200708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:10.200708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:10.200708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:00:10.200708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:00:10.200708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:10.201618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:10.202502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:00:10.202502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:00:10.202502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:00:10.202502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:00:10.202502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:00:10.202502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:00:10.202502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:10.202502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:10.209297 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 413/825] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:10.210409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bedad-4e22-436e-bac7-2ea6deb911d9 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:10.212924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77282db-5fa0-4d27-bae7-fd6f87a2e2f0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:10.235533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:10.251740 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa362f10-5e8b-4bc6-89da-62e1d279c2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:10.252347 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa362f10-5e8b-4bc6-89da-62e1d279c2ce None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:10.252666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa362f10-5e8b-4bc6-89da-62e1d279c2ce None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:10.252940 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa362f10-5e8b-4bc6-89da-62e1d279c2ce None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:10.260502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa362f10-5e8b-4bc6-89da-62e1d279c2ce None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:10.261924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa362f10-5e8b-4bc6-89da-62e1d279c2ce None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:10.274142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-fa362f10-5e8b-4bc6-89da-62e1d279c2ce None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:10.274142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:10.274142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:10.274142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:10.274142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:10.274142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:10.274142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:10.274142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:10.274142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:10.274142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:10.274142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:10.274142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:10.274142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:00:10.274142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:00:10.274142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:10.275042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:10.275932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:00:10.275932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:00:10.275932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:00:10.275932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:00:10.275932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:00:10.275932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:00:10.275932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:10.275932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:10.280233 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 413/826] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:10.281232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa362f10-5e8b-4bc6-89da-62e1d279c2ce None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:10.320345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:10.323283 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bedad-4e22-436e-bac7-2ea6deb911d9 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:10.359028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db837e0b-36f2-442a-bafd-25421075421e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:10.359759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db837e0b-36f2-442a-bafd-25421075421e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:10.360140 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db837e0b-36f2-442a-bafd-25421075421e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:10.360530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db837e0b-36f2-442a-bafd-25421075421e None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:10.361270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db837e0b-36f2-442a-bafd-25421075421e None admin] RBAC: Adding query filter params (type=network) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:10.361675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db837e0b-36f2-442a-bafd-25421075421e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:10.363175 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db837e0b-36f2-442a-bafd-25421075421e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:10.368330 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 414/827] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 19:00:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:10.373457 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db837e0b-36f2-442a-bafd-25421075421e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:10.397957 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa362f10-5e8b-4bc6-89da-62e1d279c2ce None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:10.437865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fabd5c-742d-4bae-9514-ecbb7f4878eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:10.438821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fabd5c-742d-4bae-9514-ecbb7f4878eb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:10.439330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fabd5c-742d-4bae-9514-ecbb7f4878eb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:10.439780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fabd5c-742d-4bae-9514-ecbb7f4878eb None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:10.440806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fabd5c-742d-4bae-9514-ecbb7f4878eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:10.441524 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fabd5c-742d-4bae-9514-ecbb7f4878eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:10.443134 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fabd5c-742d-4bae-9514-ecbb7f4878eb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:10.457696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 414/828] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:00:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:10.458736 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fabd5c-742d-4bae-9514-ecbb7f4878eb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:10.470244 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:10.487615 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db837e0b-36f2-442a-bafd-25421075421e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:10.522759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6396b-89a7-47f1-b5e8-887df64056af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:10.523406 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6396b-89a7-47f1-b5e8-887df64056af None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:10.523714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6396b-89a7-47f1-b5e8-887df64056af None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:10.524025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f6396b-89a7-47f1-b5e8-887df64056af None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:10.524837 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6396b-89a7-47f1-b5e8-887df64056af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:10.525253 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6396b-89a7-47f1-b5e8-887df64056af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:10.526733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f6396b-89a7-47f1-b5e8-887df64056af None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:10.545013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 415/829] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:00:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:10.546012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6396b-89a7-47f1-b5e8-887df64056af None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:10.570863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:10.578375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fabd5c-742d-4bae-9514-ecbb7f4878eb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:10.617489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7046ac8-f805-43f7-a237-bafa21a4d3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:10.618390 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7046ac8-f805-43f7-a237-bafa21a4d3f0 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:10.618845 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7046ac8-f805-43f7-a237-bafa21a4d3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:10.619320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7046ac8-f805-43f7-a237-bafa21a4d3f0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:10.620257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7046ac8-f805-43f7-a237-bafa21a4d3f0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:10.621964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7046ac8-f805-43f7-a237-bafa21a4d3f0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:10.650982 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 415/830] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:00:10 2022] POST /identity/v3/endpoints => generated 337 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:00:10.656718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f6396b-89a7-47f1-b5e8-887df64056af None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:10.658125 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7046ac8-f805-43f7-a237-bafa21a4d3f0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:10.708953 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9241ae7d-9634-420b-ad37-5c4b0a017d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:10.709680 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241ae7d-9634-420b-ad37-5c4b0a017d6a None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:10.709959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241ae7d-9634-420b-ad37-5c4b0a017d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:10.710230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9241ae7d-9634-420b-ad37-5c4b0a017d6a None admin] PATH_INFO: `/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/586715d11b924e2cb6f44030efd77619/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:10.713447 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241ae7d-9634-420b-ad37-5c4b0a017d6a None admin] RBAC: Authorizing `identity:create_grant(project_id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user_id=586715d11b924e2cb6f44030efd77619, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:10.716292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9241ae7d-9634-420b-ad37-5c4b0a017d6a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:10.745591 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:10.750802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 416/831] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 19:00:10 2022] PUT /identity/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/586715d11b924e2cb6f44030efd77619/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:00:10.840396 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7046ac8-f805-43f7-a237-bafa21a4d3f0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:10.891221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017f150d-bd8e-4278-be5b-cd67d35b3db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:10.891903 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f150d-bd8e-4278-be5b-cd67d35b3db6 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:10.892178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f150d-bd8e-4278-be5b-cd67d35b3db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:10.892451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017f150d-bd8e-4278-be5b-cd67d35b3db6 None admin] PATH_INFO: `/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/dd730375a996433d982b26c8339ec4ee/roles/6e93fc836fe74c9daa11c7b2848a2922` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:10.911507 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f150d-bd8e-4278-be5b-cd67d35b3db6 None admin] RBAC: Authorizing `identity:create_grant(project_id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user_id=dd730375a996433d982b26c8339ec4ee, role_id=6e93fc836fe74c9daa11c7b2848a2922)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:10.912967 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017f150d-bd8e-4278-be5b-cd67d35b3db6 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:10.929789 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 416/832] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 19:00:10 2022] PUT /identity/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/dd730375a996433d982b26c8339ec4ee/roles/6e93fc836fe74c9daa11c7b2848a2922 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:00:11.038191 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a54015-fc2d-41e3-99f9-c55c4037ec39 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:11.038537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a54015-fc2d-41e3-99f9-c55c4037ec39 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:11.038744 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a54015-fc2d-41e3-99f9-c55c4037ec39 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:11.040445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 417/833] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:11.046350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d2d15-1d5a-4181-8e8c-2e80d8461feb None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:11.046641 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d2d15-1d5a-4181-8e8c-2e80d8461feb None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:11.046897 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d2d15-1d5a-4181-8e8c-2e80d8461feb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:11.054551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:11.069865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-358d2d15-1d5a-4181-8e8c-2e80d8461feb None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:11.071090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-358d2d15-1d5a-4181-8e8c-2e80d8461feb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:11.147208 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 417/834] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:11.378507 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f97ee-8060-4a0d-8aa5-9996b49dc244 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:11.379018 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f97ee-8060-4a0d-8aa5-9996b49dc244 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:11.379018 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f97ee-8060-4a0d-8aa5-9996b49dc244 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:11.380598 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 418/835] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:11.385678 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6639314-6f70-494a-ade9-4e5f189fd40c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:11.386215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6639314-6f70-494a-ade9-4e5f189fd40c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:11.386822 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6639314-6f70-494a-ade9-4e5f189fd40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:11.407696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e6639314-6f70-494a-ade9-4e5f189fd40c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:11.408807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e6639314-6f70-494a-ade9-4e5f189fd40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:11.420203 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f97ee-8060-4a0d-8aa5-9996b49dc244 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:11.495072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 418/836] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:11 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:11.534898 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f97ee-8060-4a0d-8aa5-9996b49dc244 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:11.572334 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce480d0-0247-4f0d-bbfe-cd3a8528edae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:11.573208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce480d0-0247-4f0d-bbfe-cd3a8528edae None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:11.573637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce480d0-0247-4f0d-bbfe-cd3a8528edae None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:11.574152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce480d0-0247-4f0d-bbfe-cd3a8528edae None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:11.575416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce480d0-0247-4f0d-bbfe-cd3a8528edae None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:11.576981 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce480d0-0247-4f0d-bbfe-cd3a8528edae None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-1ce480d0-0247-4f0d-bbfe-cd3a8528edae None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:11.586230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:11.587420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:11.587420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:11.587420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:11.587420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:11.587420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:11.587420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:11.587420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:11.587420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:11.587420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:11.587420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:11.587420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:11.587420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:11.587420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:11.587420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:11.587420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:11.587420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:11.588492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:11.588492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:11.588492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:11.588492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:11.588492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:11.588492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:11.588492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:11.588492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:11.588492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:11.588492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:11.588492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:11.588492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:11.588492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:00:11.588492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:11.590854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 419/837] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 19:00:11 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:11.596744 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6639314-6f70-494a-ade9-4e5f189fd40c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:11.659002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:11.698925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6639314-6f70-494a-ade9-4e5f189fd40c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:11.737549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c529ec5-4215-44c5-88f1-f8789d9d7ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:11.738406 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c529ec5-4215-44c5-88f1-f8789d9d7ccb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:11.738824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c529ec5-4215-44c5-88f1-f8789d9d7ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:11.739229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c529ec5-4215-44c5-88f1-f8789d9d7ccb None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:11.739979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c529ec5-4215-44c5-88f1-f8789d9d7ccb None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:11.740412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c529ec5-4215-44c5-88f1-f8789d9d7ccb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:11.741925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c529ec5-4215-44c5-88f1-f8789d9d7ccb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:11.747420 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 419/838] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 19:00:11 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:11.754548 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce480d0-0247-4f0d-bbfe-cd3a8528edae None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:11.785290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c529ec5-4215-44c5-88f1-f8789d9d7ccb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:11.819984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:11.864140 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce480d0-0247-4f0d-bbfe-cd3a8528edae None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:11.901262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c03eb-0011-4840-8e7e-669d4e7bf94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:11.901871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c03eb-0011-4840-8e7e-669d4e7bf94f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:11.902193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c03eb-0011-4840-8e7e-669d4e7bf94f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:11.902533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0c03eb-0011-4840-8e7e-669d4e7bf94f None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:11.903224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c03eb-0011-4840-8e7e-669d4e7bf94f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:11.903458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c03eb-0011-4840-8e7e-669d4e7bf94f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:11.904811 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0c03eb-0011-4840-8e7e-669d4e7bf94f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:11.906770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c529ec5-4215-44c5-88f1-f8789d9d7ccb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:11.917380 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 420/839] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 19:00:11 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:11.940317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:11.952591 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a18ec-eaf6-402c-8585-89272001f57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:11.953584 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a18ec-eaf6-402c-8585-89272001f57a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:11.954018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a18ec-eaf6-402c-8585-89272001f57a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:11.954427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a18ec-eaf6-402c-8585-89272001f57a None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:11.955160 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a18ec-eaf6-402c-8585-89272001f57a None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:11.956770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a18ec-eaf6-402c-8585-89272001f57a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:11.967404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-fe6a18ec-eaf6-402c-8585-89272001f57a None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:00:11.967404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:11.967404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:11.967404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:11.967404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:11.967404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:11.967404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:11.967404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:11.967404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:00:11.967404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:11.967404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:11.967404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:11.967404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:11.967404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:11.967404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:11.967404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:11.968605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:11.969725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:11.969725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:11.969725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:11.969725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:11.969725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:11.969725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:11.969725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:11.969725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:11.969725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:11.969725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:11.969725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:11.969725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:11.969725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:00:11.969725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:11.972244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 420/840] 10.222.0.65 () {56 vars in 1132 bytes} [Wed Aug 31 19:00:11 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:11.978144 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c03eb-0011-4840-8e7e-669d4e7bf94f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:12.099421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0c03eb-0011-4840-8e7e-669d4e7bf94f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:12.127567 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:12.137147 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea074e1-64d9-4694-80db-56c478ee949f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:12.138022 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea074e1-64d9-4694-80db-56c478ee949f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:12.138466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea074e1-64d9-4694-80db-56c478ee949f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:12.138869 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea074e1-64d9-4694-80db-56c478ee949f None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:12.139624 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea074e1-64d9-4694-80db-56c478ee949f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:12.140060 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea074e1-64d9-4694-80db-56c478ee949f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:12.141570 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea074e1-64d9-4694-80db-56c478ee949f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:12.147139 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 421/841] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 19:00:11 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:12.154364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a18ec-eaf6-402c-8585-89272001f57a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:12.270085 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a18ec-eaf6-402c-8585-89272001f57a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:12.306047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9d91a0-cea2-4faf-8815-28950b01c957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:12.306913 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d91a0-cea2-4faf-8815-28950b01c957 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:12.307329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d91a0-cea2-4faf-8815-28950b01c957 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:12.307735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9d91a0-cea2-4faf-8815-28950b01c957 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:12.308528 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d91a0-cea2-4faf-8815-28950b01c957 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:12.308969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d91a0-cea2-4faf-8815-28950b01c957 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:12.310486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9d91a0-cea2-4faf-8815-28950b01c957 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:12.315509 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 421/842] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 19:00:12 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:12.783421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf78906-50cb-42c5-b697-20f5cb855197 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:12.783968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf78906-50cb-42c5-b697-20f5cb855197 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:12.783968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf78906-50cb-42c5-b697-20f5cb855197 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:12.785569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 422/843] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:12.790776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d85463-3da9-42bd-9673-ea2b1aa08da5 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:12.791061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d85463-3da9-42bd-9673-ea2b1aa08da5 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:12.791329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d85463-3da9-42bd-9673-ea2b1aa08da5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:12.810052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e6d85463-3da9-42bd-9673-ea2b1aa08da5 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:12.811235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e6d85463-3da9-42bd-9673-ea2b1aa08da5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:12.861502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:12.885317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 422/844] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:12 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:13.054789 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e1e69-ff79-40f2-9d21-b650c70f43da None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:13.055411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e1e69-ff79-40f2-9d21-b650c70f43da None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:13.055879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e1e69-ff79-40f2-9d21-b650c70f43da None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:13.057689 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 423/845] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:13.063353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b195a-1eba-4660-a8de-931e56f59248 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:13.064168 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b195a-1eba-4660-a8de-931e56f59248 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:13.064647 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b195a-1eba-4660-a8de-931e56f59248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:13.083947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-767b195a-1eba-4660-a8de-931e56f59248 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:13.085534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-767b195a-1eba-4660-a8de-931e56f59248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:13.152794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e1e69-ff79-40f2-9d21-b650c70f43da None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:13.155233 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 423/846] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:13.259170 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e1e69-ff79-40f2-9d21-b650c70f43da None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:13.298043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f90936-f7d8-44da-bbd4-dd29a730fe01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:13.298668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f90936-f7d8-44da-bbd4-dd29a730fe01 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:13.298948 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f90936-f7d8-44da-bbd4-dd29a730fe01 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:13.299214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f90936-f7d8-44da-bbd4-dd29a730fe01 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:13.310828 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f90936-f7d8-44da-bbd4-dd29a730fe01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:00:13.315514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-f6f90936-f7d8-44da-bbd4-dd29a730fe01 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:00:13.315514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:13.315514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:13.315514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:13.315514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:13.315514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:13.315514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:13.315514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:13.315514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:13.315514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:13.315514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:13.315514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:13.315514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:00:13.315514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:00:13.315514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:00:13.315514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:00:13.316409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:00:13.316409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:00:13.316409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:00:13.316409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:00:13.316409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:13.316409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:13.316409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:13.316409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:13.316409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:13.316409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:13.316409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:13.316409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:13.316409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:13.316409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:13.316409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:13.316409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:13.317636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:13.317636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:13.317636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:13.317636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:13.317636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:00:13.317636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:00:13.317636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:00:13.317636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:00:13.317636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:00:13.317636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:00:13.317636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:00:13.317636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:13.317636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 424/847] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:00:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:13.324616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b195a-1eba-4660-a8de-931e56f59248 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:13.433014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b195a-1eba-4660-a8de-931e56f59248 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:13.447238 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f90936-f7d8-44da-bbd4-dd29a730fe01 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:13.448314 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:13.475110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847321e-6a13-46d6-b7e8-c28c26a33b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:13.475999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847321e-6a13-46d6-b7e8-c28c26a33b9e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:13.476282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847321e-6a13-46d6-b7e8-c28c26a33b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:13.476555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847321e-6a13-46d6-b7e8-c28c26a33b9e None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:13.477417 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847321e-6a13-46d6-b7e8-c28c26a33b9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:13.478047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847321e-6a13-46d6-b7e8-c28c26a33b9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:13.479688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847321e-6a13-46d6-b7e8-c28c26a33b9e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:13.491514 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 424/848] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:00:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:13.496995 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847321e-6a13-46d6-b7e8-c28c26a33b9e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:13.514602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:13.571400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f90936-f7d8-44da-bbd4-dd29a730fe01 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:13.617637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dec19b-1158-413e-9b9f-f414026298fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:13.618394 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dec19b-1158-413e-9b9f-f414026298fd None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:13.618798 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dec19b-1158-413e-9b9f-f414026298fd None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:13.619130 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dec19b-1158-413e-9b9f-f414026298fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:13.627045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847321e-6a13-46d6-b7e8-c28c26a33b9e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:13.632870 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dec19b-1158-413e-9b9f-f414026298fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:00:13.639845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-e9dec19b-1158-413e-9b9f-f414026298fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:00:13.639845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:13.639845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:13.639845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:13.639845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:13.639845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:13.639845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:13.639845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:13.639845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:13.639845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:13.639845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:13.639845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:13.639845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:00:13.639845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:00:13.639845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:00:13.639845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:00:13.640771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:00:13.640771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:00:13.640771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:00:13.640771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:00:13.640771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:13.640771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:13.640771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:13.640771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:13.640771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:13.640771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:13.640771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:13.640771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:13.640771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:13.640771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:13.640771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:13.640771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:13.641648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:13.641648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:13.641648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:13.641648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:13.641648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:00:13.641648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:00:13.641648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:00:13.641648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:00:13.641648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:00:13.641648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:00:13.641648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:00:13.641648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:13.645265 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 425/849] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 19:00:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:13.647572 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dec19b-1158-413e-9b9f-f414026298fd None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:13.670133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36a9a0-3f91-465e-864d-3adc965e3979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:13.671574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36a9a0-3f91-465e-864d-3adc965e3979 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:13.672238 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36a9a0-3f91-465e-864d-3adc965e3979 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:13.672944 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb36a9a0-3f91-465e-864d-3adc965e3979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:13.684671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36a9a0-3f91-465e-864d-3adc965e3979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:13.686532 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb36a9a0-3f91-465e-864d-3adc965e3979 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:13.701550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-fb36a9a0-3f91-465e-864d-3adc965e3979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:13.701550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:13.701550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:13.701550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:13.701550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:13.701550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:13.701550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:13.701550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:13.701550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:13.701550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:13.701550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:13.701550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:13.701550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:13.701550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:13.701550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:13.701550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:13.702510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:13.703398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:13.703398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:13.703398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:13.705442 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 425/850] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:13.712502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36a9a0-3f91-465e-864d-3adc965e3979 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:13.750766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:13.780940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dec19b-1158-413e-9b9f-f414026298fd None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:13.798883 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:13.827747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f435a5-a311-4c2e-a647-c7d48188a353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:13.828316 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f435a5-a311-4c2e-a647-c7d48188a353 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:13.828596 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f435a5-a311-4c2e-a647-c7d48188a353 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:13.828864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f435a5-a311-4c2e-a647-c7d48188a353 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:13.829933 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f435a5-a311-4c2e-a647-c7d48188a353 None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:13.830237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f435a5-a311-4c2e-a647-c7d48188a353 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:13.831590 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f435a5-a311-4c2e-a647-c7d48188a353 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:13.842838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 426/851] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 19:00:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:13.848653 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f435a5-a311-4c2e-a647-c7d48188a353 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:13.853988 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb36a9a0-3f91-465e-864d-3adc965e3979 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:13.899366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f1b09c-b7ed-43e7-b8bf-cf0e220df09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:13.900352 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1b09c-b7ed-43e7-b8bf-cf0e220df09a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:13.900823 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1b09c-b7ed-43e7-b8bf-cf0e220df09a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:13.901310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f1b09c-b7ed-43e7-b8bf-cf0e220df09a None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:13.902789 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1b09c-b7ed-43e7-b8bf-cf0e220df09a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:13.903298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1b09c-b7ed-43e7-b8bf-cf0e220df09a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:13.905277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f1b09c-b7ed-43e7-b8bf-cf0e220df09a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:13.920334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 426/852] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:13.921981 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304f9a3-7a32-4713-8d7d-5ceab17f63b4 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:13.922472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304f9a3-7a32-4713-8d7d-5ceab17f63b4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:13.922800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c304f9a3-7a32-4713-8d7d-5ceab17f63b4 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:13.924065 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 427/853] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:13.926786 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304f9a3-7a32-4713-8d7d-5ceab17f63b4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:13.972377 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f435a5-a311-4c2e-a647-c7d48188a353 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:14.005793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb41073-6ec3-48d3-b1eb-a5f1047f86ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:14.007248 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb41073-6ec3-48d3-b1eb-a5f1047f86ae None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:14.007523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb41073-6ec3-48d3-b1eb-a5f1047f86ae None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:14.007785 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb41073-6ec3-48d3-b1eb-a5f1047f86ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:14.020311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb41073-6ec3-48d3-b1eb-a5f1047f86ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:14.021760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb41073-6ec3-48d3-b1eb-a5f1047f86ae None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:14.025925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:14.037160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-ceb41073-6ec3-48d3-b1eb-a5f1047f86ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:14.037160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:14.037160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:14.037160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:14.037160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:14.037160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:14.037160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:14.037160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:14.037160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:14.037160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:14.037160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:14.037160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:14.037160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:14.037160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:14.037160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:14.037160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:14.038125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:14.039044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:14.039044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:14.039044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:14.042579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c304f9a3-7a32-4713-8d7d-5ceab17f63b4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:14.042983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 427/854] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:14.044291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b26b9-9a93-4ed7-a475-cad88aafca39 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:14.044577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b26b9-9a93-4ed7-a475-cad88aafca39 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:14.044843 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197b26b9-9a93-4ed7-a475-cad88aafca39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:14.063799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-197b26b9-9a93-4ed7-a475-cad88aafca39 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:14.064970 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-197b26b9-9a93-4ed7-a475-cad88aafca39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:14.080843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a816d-9432-4e28-97f7-b9639f2415a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:14.082076 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a816d-9432-4e28-97f7-b9639f2415a5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:14.082532 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a816d-9432-4e28-97f7-b9639f2415a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:14.082978 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9a816d-9432-4e28-97f7-b9639f2415a5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:14.086253 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:14.092163 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a816d-9432-4e28-97f7-b9639f2415a5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:14.093680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9a816d-9432-4e28-97f7-b9639f2415a5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:14.105488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-7a9a816d-9432-4e28-97f7-b9639f2415a5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:00:14.105488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:14.105488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:14.105488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:14.105488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:14.105488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:14.105488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:14.105488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:14.105488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:14.105488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:14.105488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:14.105488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:14.105488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:00:14.105488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:00:14.105488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:00:14.105488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:14.106466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:14.112768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:14.112768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:14.112768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:00:14.112768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:00:14.112768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:00:14.112768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:00:14.112768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:00:14.112768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:00:14.112768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 31 19:00:14.112768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:14.112768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 428/855] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:00:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:14.112768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a816d-9432-4e28-97f7-b9639f2415a5 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:14.139460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 428/856] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:14.140320 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b26b9-9a93-4ed7-a475-cad88aafca39 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:14.245530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9a816d-9432-4e28-97f7-b9639f2415a5 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:14.274683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197b26b9-9a93-4ed7-a475-cad88aafca39 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:14.282247 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ce0e5-be19-44cd-b790-363576094f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:14.282847 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ce0e5-be19-44cd-b790-363576094f81 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:14.283121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ce0e5-be19-44cd-b790-363576094f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:14.283388 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ce0e5-be19-44cd-b790-363576094f81 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:14.284173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ce0e5-be19-44cd-b790-363576094f81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:14.284473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ce0e5-be19-44cd-b790-363576094f81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:14.286041 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ce0e5-be19-44cd-b790-363576094f81 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:14.301369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 429/857] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:14.301369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71017cc-af34-43fb-8f8f-07c5f2f425a7 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:14.301369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71017cc-af34-43fb-8f8f-07c5f2f425a7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:14.301369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71017cc-af34-43fb-8f8f-07c5f2f425a7 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:14.302151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 430/858] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:14.304075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71017cc-af34-43fb-8f8f-07c5f2f425a7 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:14.311583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3861fb46-9d69-43e5-b960-4a050f6411ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:14.312188 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861fb46-9d69-43e5-b960-4a050f6411ea None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:14.312467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861fb46-9d69-43e5-b960-4a050f6411ea None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:14.312740 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3861fb46-9d69-43e5-b960-4a050f6411ea None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:14.313506 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861fb46-9d69-43e5-b960-4a050f6411ea None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:14.313810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861fb46-9d69-43e5-b960-4a050f6411ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:14.315227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3861fb46-9d69-43e5-b960-4a050f6411ea None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:14.316345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:14.338123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 429/859] 10.222.0.65 () {56 vars in 1135 bytes} [Wed Aug 31 19:00:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:14.338123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bd5d5-1f59-4389-8b67-42696b98e968 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:14.338123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bd5d5-1f59-4389-8b67-42696b98e968 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:14.338123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1bd5d5-1f59-4389-8b67-42696b98e968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:14.353326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4a1bd5d5-1f59-4389-8b67-42696b98e968 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:14.354520 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4a1bd5d5-1f59-4389-8b67-42696b98e968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:14.363772 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:14.424592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 430/860] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:14.425311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bd5d5-1f59-4389-8b67-42696b98e968 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:14.425568 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71017cc-af34-43fb-8f8f-07c5f2f425a7 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:14.461121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606b972-a286-46fb-877f-6cd2af4f6bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:14.461775 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606b972-a286-46fb-877f-6cd2af4f6bbb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:14.462053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606b972-a286-46fb-877f-6cd2af4f6bbb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:14.462326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9606b972-a286-46fb-877f-6cd2af4f6bbb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:14.469587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606b972-a286-46fb-877f-6cd2af4f6bbb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:14.471011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9606b972-a286-46fb-877f-6cd2af4f6bbb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:14.482554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-9606b972-a286-46fb-877f-6cd2af4f6bbb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:00:14.482554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:14.482554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:14.482554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:14.482554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:14.482554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:14.482554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:14.482554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:14.482554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:14.482554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:14.482554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:14.482554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:14.482554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:00:14.482554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:00:14.482554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:00:14.482554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:14.483487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:14.484492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:14.484492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:14.484492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:00:14.484492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:00:14.484492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:00:14.484492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:00:14.484492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:00:14.484492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:00:14.484492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:00:14.484492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:14.484492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 431/861] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 19:00:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:14.484492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606b972-a286-46fb-877f-6cd2af4f6bbb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:14.533995 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1bd5d5-1f59-4389-8b67-42696b98e968 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:14.568880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d1f9b-03ed-4e5d-86ce-b14d16c84c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:14.569605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d1f9b-03ed-4e5d-86ce-b14d16c84c87 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:14.569885 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d1f9b-03ed-4e5d-86ce-b14d16c84c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:14.570155 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d1f9b-03ed-4e5d-86ce-b14d16c84c87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:14.573471 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:14.586229 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:14.616177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9606b972-a286-46fb-877f-6cd2af4f6bbb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:14.642835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d1f9b-03ed-4e5d-86ce-b14d16c84c87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:14.644387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2d1f9b-03ed-4e5d-86ce-b14d16c84c87 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:14.652381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a23bb9-8493-4f9f-9c7a-e7aa3573f216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:14.653020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a23bb9-8493-4f9f-9c7a-e7aa3573f216 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:14.653311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a23bb9-8493-4f9f-9c7a-e7aa3573f216 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:14.653557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a23bb9-8493-4f9f-9c7a-e7aa3573f216 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:14.654200 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a23bb9-8493-4f9f-9c7a-e7aa3573f216 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:14.655547 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a23bb9-8493-4f9f-9c7a-e7aa3573f216 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:14.676769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 432/862] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 19:00:14 2022] POST /identity/v3/services => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:00:14.677455 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a23bb9-8493-4f9f-9c7a-e7aa3573f216 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:14.775050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 431/863] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:00:14 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:00:14.775910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d1f9b-03ed-4e5d-86ce-b14d16c84c87 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:14.812810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a23bb9-8493-4f9f-9c7a-e7aa3573f216 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:14.830927 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:14.841274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:14.857589 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65214041-de63-49ab-ade6-2616d0fbf1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:14.858481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65214041-de63-49ab-ade6-2616d0fbf1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:14.858760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65214041-de63-49ab-ade6-2616d0fbf1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:14.859037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65214041-de63-49ab-ade6-2616d0fbf1f2 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:14.859782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65214041-de63-49ab-ade6-2616d0fbf1f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:14.860084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65214041-de63-49ab-ade6-2616d0fbf1f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:14.861549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65214041-de63-49ab-ade6-2616d0fbf1f2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:14.880766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 433/864] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 31 19:00:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:14.881425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65214041-de63-49ab-ade6-2616d0fbf1f2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:14.894011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d1f9b-03ed-4e5d-86ce-b14d16c84c87 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:14.931575 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4d41d-2d94-4284-9a95-e81491638046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:14.932194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4d41d-2d94-4284-9a95-e81491638046 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:14.932459 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4d41d-2d94-4284-9a95-e81491638046 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:14.932730 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea4d41d-2d94-4284-9a95-e81491638046 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:14.934001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4d41d-2d94-4284-9a95-e81491638046 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:14.935325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea4d41d-2d94-4284-9a95-e81491638046 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:14.953255 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 432/865] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 19:00:14 2022] POST /identity/v3/services => generated 269 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:00:14.953255 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4d41d-2d94-4284-9a95-e81491638046 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:15.016858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65214041-de63-49ab-ade6-2616d0fbf1f2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:15.074241 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3beb8-7371-427e-b885-215323f0ea5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:15.075393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3beb8-7371-427e-b885-215323f0ea5e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:15.076268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3beb8-7371-427e-b885-215323f0ea5e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:15.076731 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e3beb8-7371-427e-b885-215323f0ea5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:15.091714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3beb8-7371-427e-b885-215323f0ea5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:15.094535 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e3beb8-7371-427e-b885-215323f0ea5e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:15.099515 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea4d41d-2d94-4284-9a95-e81491638046 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:15.102655 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:15.118027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-19e3beb8-7371-427e-b885-215323f0ea5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:15.118027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:15.118027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:15.118027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:15.118027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:15.118027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:15.118027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:15.118027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:15.118027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:15.118027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:15.118027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:15.118027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:15.118027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:15.118027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:15.118027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:15.118027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:15.119074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:15.120647 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:15.120647 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:15.120647 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:15.120647 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 434/866] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:15.125174 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3beb8-7371-427e-b885-215323f0ea5e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:15.143482 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49733dd2-4936-4063-b1d7-d612e0ceb317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:15.144021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49733dd2-4936-4063-b1d7-d612e0ceb317 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:15.144301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49733dd2-4936-4063-b1d7-d612e0ceb317 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:15.144575 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49733dd2-4936-4063-b1d7-d612e0ceb317 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:15.169473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:15.231417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49733dd2-4936-4063-b1d7-d612e0ceb317 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:15.232866 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49733dd2-4936-4063-b1d7-d612e0ceb317 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:15.266570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e3beb8-7371-427e-b885-215323f0ea5e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:15.304788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507ef4c-d93e-453e-bd89-323249019d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:15.305782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507ef4c-d93e-453e-bd89-323249019d2e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:15.306270 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507ef4c-d93e-453e-bd89-323249019d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:15.306770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e507ef4c-d93e-453e-bd89-323249019d2e None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:15.307614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507ef4c-d93e-453e-bd89-323249019d2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:15.308175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507ef4c-d93e-453e-bd89-323249019d2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:15.309959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e507ef4c-d93e-453e-bd89-323249019d2e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:15.324491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 435/867] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:15.332232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507ef4c-d93e-453e-bd89-323249019d2e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:15.336342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 433/868] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:00:14 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:00:15.343306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49733dd2-4936-4063-b1d7-d612e0ceb317 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:15.346915 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:15.436165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:15.447445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49733dd2-4936-4063-b1d7-d612e0ceb317 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:15.451379 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e507ef4c-d93e-453e-bd89-323249019d2e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:15.493275 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb15fd-0564-4ec1-9151-9c60b41b3ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:15.494114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb15fd-0564-4ec1-9151-9c60b41b3ba7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:15.494468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb15fd-0564-4ec1-9151-9c60b41b3ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:15.494980 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcb15fd-0564-4ec1-9151-9c60b41b3ba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:15.497421 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e10560-88d0-4c6a-a013-c2ba0e341ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:15.498683 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e10560-88d0-4c6a-a013-c2ba0e341ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:15.499395 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e10560-88d0-4c6a-a013-c2ba0e341ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:15.499843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e10560-88d0-4c6a-a013-c2ba0e341ef3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:15.507321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb15fd-0564-4ec1-9151-9c60b41b3ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:15.508759 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e10560-88d0-4c6a-a013-c2ba0e341ef3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:15.509230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcb15fd-0564-4ec1-9151-9c60b41b3ba7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:15.510599 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e10560-88d0-4c6a-a013-c2ba0e341ef3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:15.524882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-a7e10560-88d0-4c6a-a013-c2ba0e341ef3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:15.524882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:15.524882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:15.524882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:15.524882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:15.524882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:15.524882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:15.524882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:15.524882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:15.524882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:15.524882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:15.524882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:15.524882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:00:15.524882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:00:15.524882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:15.525813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:15.526712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:00:15.526712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:00:15.526712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:00:15.526712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:00:15.526712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:00:15.526712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:00:15.526712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:15.526712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:15.527781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-7fcb15fd-0564-4ec1-9151-9c60b41b3ba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:15.527781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:15.527781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:15.527781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:15.527781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:15.527781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:15.527781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:15.527781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:15.527781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:15.527781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:15.527781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:15.527781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:15.527781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:15.527781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:15.527781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:15.527781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:15.528641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:15.529552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:15.529552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:15.529552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:15.529552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 434/869] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:15.530939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 436/870] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:15.536156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb15fd-0564-4ec1-9151-9c60b41b3ba7 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:15.537501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e10560-88d0-4c6a-a013-c2ba0e341ef3 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:15.601034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:15.644168 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e10560-88d0-4c6a-a013-c2ba0e341ef3 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:15.649023 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcb15fd-0564-4ec1-9151-9c60b41b3ba7 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:15.680904 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a0e28-1f63-43bc-b61b-2a1f4ba53ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:15.681817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a0e28-1f63-43bc-b61b-2a1f4ba53ee1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:15.682105 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a0e28-1f63-43bc-b61b-2a1f4ba53ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:15.682378 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a0e28-1f63-43bc-b61b-2a1f4ba53ee1 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:15.683066 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a0e28-1f63-43bc-b61b-2a1f4ba53ee1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:15.683401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a0e28-1f63-43bc-b61b-2a1f4ba53ee1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:15.684832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622a0e28-1f63-43bc-b61b-2a1f4ba53ee1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:15.686565 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e490c10-b121-42c8-8973-2f77dfeabc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:15.687343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e490c10-b121-42c8-8973-2f77dfeabc1d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:15.687765 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e490c10-b121-42c8-8973-2f77dfeabc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:15.688191 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e490c10-b121-42c8-8973-2f77dfeabc1d None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:15.688982 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e490c10-b121-42c8-8973-2f77dfeabc1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:15.689468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e490c10-b121-42c8-8973-2f77dfeabc1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:15.690982 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e490c10-b121-42c8-8973-2f77dfeabc1d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:15.701284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 437/871] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:00:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:15.706092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a0e28-1f63-43bc-b61b-2a1f4ba53ee1 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:15.713559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 435/872] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:15.720397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e490c10-b121-42c8-8973-2f77dfeabc1d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:15.722196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:15.837775 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e490c10-b121-42c8-8973-2f77dfeabc1d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:15.856080 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a0e28-1f63-43bc-b61b-2a1f4ba53ee1 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:15.857351 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:15.883058 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec9bdd-c78d-41dd-8d03-c23cfbd6a177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:15.884336 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec9bdd-c78d-41dd-8d03-c23cfbd6a177 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:15.884816 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec9bdd-c78d-41dd-8d03-c23cfbd6a177 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:15.885551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec9bdd-c78d-41dd-8d03-c23cfbd6a177 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:15.896047 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec9bdd-c78d-41dd-8d03-c23cfbd6a177 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:15.897798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba076f-6d21-45fe-b86a-2c853002821e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:15.898101 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec9bdd-c78d-41dd-8d03-c23cfbd6a177 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:15.898398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba076f-6d21-45fe-b86a-2c853002821e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:15.898677 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba076f-6d21-45fe-b86a-2c853002821e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:15.898945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ba076f-6d21-45fe-b86a-2c853002821e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:15.899852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba076f-6d21-45fe-b86a-2c853002821e None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=586715d11b924e2cb6f44030efd77619) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:15.900157 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba076f-6d21-45fe-b86a-2c853002821e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:15.901603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ba076f-6d21-45fe-b86a-2c853002821e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:15.912285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-6bec9bdd-c78d-41dd-8d03-c23cfbd6a177 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:15.912285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:15.912285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:15.912285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:15.912285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:15.912285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:15.912285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:15.912285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:15.912285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:15.912285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:15.912285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:15.912285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:15.912285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:00:15.912285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:00:15.912285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:15.913254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:15.914137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:00:15.914137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:00:15.914137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:00:15.914137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:00:15.914137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:00:15.914137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:00:15.914137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:15.914137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:15.914570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 438/873] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:00:15 2022] GET /identity/v3/role_assignments?user.id=586715d11b924e2cb6f44030efd77619&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=3580c894dd974d35bc95848b8ba22598 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:15.921431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 436/874] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:15.923175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba076f-6d21-45fe-b86a-2c853002821e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:16.010413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:16.040906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ba076f-6d21-45fe-b86a-2c853002821e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:16.078394 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3be2cf5-d531-4b81-bfbb-4bb6a8730526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:16.079007 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be2cf5-d531-4b81-bfbb-4bb6a8730526 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:16.079291 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be2cf5-d531-4b81-bfbb-4bb6a8730526 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:16.079745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3be2cf5-d531-4b81-bfbb-4bb6a8730526 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:16.080438 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be2cf5-d531-4b81-bfbb-4bb6a8730526 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:16.080739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be2cf5-d531-4b81-bfbb-4bb6a8730526 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:16.082191 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3be2cf5-d531-4b81-bfbb-4bb6a8730526 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:16.095255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 439/875] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:00:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:16.101145 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec9bdd-c78d-41dd-8d03-c23cfbd6a177 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:16.212419 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec9bdd-c78d-41dd-8d03-c23cfbd6a177 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:16.245350 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e25a987-487e-4f09-ab36-72edcea20f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:16.246498 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25a987-487e-4f09-ab36-72edcea20f1a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:16.246787 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25a987-487e-4f09-ab36-72edcea20f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:16.247107 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e25a987-487e-4f09-ab36-72edcea20f1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:16.247834 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25a987-487e-4f09-ab36-72edcea20f1a None admin] RBAC: Adding query filter params (role.id=6e93fc836fe74c9daa11c7b2848a2922, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=dd730375a996433d982b26c8339ec4ee) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:16.248138 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25a987-487e-4f09-ab36-72edcea20f1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:16.249842 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e25a987-487e-4f09-ab36-72edcea20f1a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:16.256326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 437/876] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:00:16 2022] GET /identity/v3/role_assignments?user.id=dd730375a996433d982b26c8339ec4ee&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=6e93fc836fe74c9daa11c7b2848a2922 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:16.641808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1e3a2-1dda-4b91-9435-b8eb9792b381 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:16.642228 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1e3a2-1dda-4b91-9435-b8eb9792b381 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:16.642408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc1e3a2-1dda-4b91-9435-b8eb9792b381 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:16.643853 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 440/877] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:16.649372 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78d639-917b-4f07-a5e1-31a2a2872363 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:16.649919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78d639-917b-4f07-a5e1-31a2a2872363 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:16.650396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec78d639-917b-4f07-a5e1-31a2a2872363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:16.670348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ec78d639-917b-4f07-a5e1-31a2a2872363 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:16.671842 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ec78d639-917b-4f07-a5e1-31a2a2872363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:16.707969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:16.748042 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 438/878] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:16.959308 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211969d-a230-4d9a-8318-6df5dcc51cbc None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:16.959652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211969d-a230-4d9a-8318-6df5dcc51cbc None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:16.959755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3211969d-a230-4d9a-8318-6df5dcc51cbc None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:16.961384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 441/879] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:16.966992 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a0ddb-e036-4944-b134-208fe777900f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:16.967841 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a0ddb-e036-4944-b134-208fe777900f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:16.968307 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67a0ddb-e036-4944-b134-208fe777900f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:16.986660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a67a0ddb-e036-4944-b134-208fe777900f None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:16.988128 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a67a0ddb-e036-4944-b134-208fe777900f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:17.047205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211969d-a230-4d9a-8318-6df5dcc51cbc None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:17.068109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 439/880] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:17.159516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3211969d-a230-4d9a-8318-6df5dcc51cbc None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:17.197218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e3542-8886-4aac-b5cb-41eacb2485bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:17.197218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e3542-8886-4aac-b5cb-41eacb2485bb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:17.197218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e3542-8886-4aac-b5cb-41eacb2485bb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:17.197218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e3542-8886-4aac-b5cb-41eacb2485bb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:17.197572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e3542-8886-4aac-b5cb-41eacb2485bb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:17.198899 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e3542-8886-4aac-b5cb-41eacb2485bb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:17.200379 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-754e3542-8886-4aac-b5cb-41eacb2485bb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:17.208090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:17.209036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:17.209036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:17.209036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:17.209036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:17.209036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:17.209036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:17.209036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:17.209036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:17.209036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:17.209036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:17.209036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:17.209036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:17.209036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:17.209036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:17.209036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:17.209036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:17.209947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:17.209947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:17.209947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:17.209947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:17.209947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:17.209947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:17.209947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:17.209947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:17.209947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:17.209947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:17.209947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:17.209947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:17.209947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:00:17.209947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:17.214466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 442/881] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 19:00:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:17.218281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a0ddb-e036-4944-b134-208fe777900f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:17.332997 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:17.337239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67a0ddb-e036-4944-b134-208fe777900f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:17.352423 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e3542-8886-4aac-b5cb-41eacb2485bb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:17.398273 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6abd12-c76c-482d-ab22-b9f122148ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:17.399823 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6abd12-c76c-482d-ab22-b9f122148ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:17.400324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6abd12-c76c-482d-ab22-b9f122148ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:17.401300 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6abd12-c76c-482d-ab22-b9f122148ed3 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:17.402553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6abd12-c76c-482d-ab22-b9f122148ed3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:17.403054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6abd12-c76c-482d-ab22-b9f122148ed3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:17.406429 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6abd12-c76c-482d-ab22-b9f122148ed3 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:17.420600 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 440/882] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 19:00:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:17.422500 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6abd12-c76c-482d-ab22-b9f122148ed3 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:17.469487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e3542-8886-4aac-b5cb-41eacb2485bb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:17.504483 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a893ab-f42e-4168-9d9b-847bb6e0bcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:17.505199 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a893ab-f42e-4168-9d9b-847bb6e0bcc9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:17.505622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a893ab-f42e-4168-9d9b-847bb6e0bcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:17.505956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a893ab-f42e-4168-9d9b-847bb6e0bcc9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:17.506594 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a893ab-f42e-4168-9d9b-847bb6e0bcc9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:17.507998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a893ab-f42e-4168-9d9b-847bb6e0bcc9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:17.517592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-04a893ab-f42e-4168-9d9b-847bb6e0bcc9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:00:17.517592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:17.517592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:17.517592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:17.517592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:17.517592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:17.517592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:17.517592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:17.517592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:00:17.517592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:17.517592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:17.517592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:17.517592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:17.517592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:17.517592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:17.517592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:17.518567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:17.519753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:17.519753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:17.519753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:17.519753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:17.519753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:17.519753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:17.519753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:17.519753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:17.519753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:17.519753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:17.519753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:17.519753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:17.519753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:00:17.519753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:17.523701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 443/883] 10.222.0.65 () {56 vars in 1132 bytes} [Wed Aug 31 19:00:17 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:17.524619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a893ab-f42e-4168-9d9b-847bb6e0bcc9 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:17.546094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6abd12-c76c-482d-ab22-b9f122148ed3 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:17.586854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0d52b-3e3a-4e71-b1df-dc4c5eafdde3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:17.587590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:17.589628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0d52b-3e3a-4e71-b1df-dc4c5eafdde3 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:17.590241 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0d52b-3e3a-4e71-b1df-dc4c5eafdde3 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:17.590711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d0d52b-3e3a-4e71-b1df-dc4c5eafdde3 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:17.591528 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0d52b-3e3a-4e71-b1df-dc4c5eafdde3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:17.592003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0d52b-3e3a-4e71-b1df-dc4c5eafdde3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:17.595257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d0d52b-3e3a-4e71-b1df-dc4c5eafdde3 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:17.607779 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 441/884] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 19:00:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:17.610371 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0d52b-3e3a-4e71-b1df-dc4c5eafdde3 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:17.635219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a893ab-f42e-4168-9d9b-847bb6e0bcc9 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:17.655968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:17.674006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea51c22-ec50-4a77-8823-3806650a3552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:17.675155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea51c22-ec50-4a77-8823-3806650a3552 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:17.675617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea51c22-ec50-4a77-8823-3806650a3552 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:17.676083 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea51c22-ec50-4a77-8823-3806650a3552 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:17.677214 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea51c22-ec50-4a77-8823-3806650a3552 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:17.677765 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea51c22-ec50-4a77-8823-3806650a3552 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:17.680240 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea51c22-ec50-4a77-8823-3806650a3552 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:17.692454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 444/885] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 19:00:17 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:17.693532 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea51c22-ec50-4a77-8823-3806650a3552 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:17.768784 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d0d52b-3e3a-4e71-b1df-dc4c5eafdde3 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:17.812181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93b106-43b0-41ba-bd24-33c602b9b871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:17.812787 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93b106-43b0-41ba-bd24-33c602b9b871 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:17.813060 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93b106-43b0-41ba-bd24-33c602b9b871 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:17.813376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93b106-43b0-41ba-bd24-33c602b9b871 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:17.813652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea51c22-ec50-4a77-8823-3806650a3552 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:17.814066 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93b106-43b0-41ba-bd24-33c602b9b871 None admin] RBAC: Adding query filter params (interface=public, service_id=8b336a6aa5c84baea9daefcf298abb76, region_id=RegionOne) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:17.814365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93b106-43b0-41ba-bd24-33c602b9b871 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:17.815783 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f93b106-43b0-41ba-bd24-33c602b9b871 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:17.821226 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 442/886] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 31 19:00:17 2022] GET /identity/v3/endpoints?service_id=8b336a6aa5c84baea9daefcf298abb76&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:17.826718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93b106-43b0-41ba-bd24-33c602b9b871 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:17.847830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:17.858681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1966be-b25a-4e9a-b941-9a5fd74c9859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:17.859535 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1966be-b25a-4e9a-b941-9a5fd74c9859 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:17.860005 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1966be-b25a-4e9a-b941-9a5fd74c9859 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:17.860460 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1966be-b25a-4e9a-b941-9a5fd74c9859 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:17.861288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1966be-b25a-4e9a-b941-9a5fd74c9859 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:17.861844 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1966be-b25a-4e9a-b941-9a5fd74c9859 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:17.863475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1966be-b25a-4e9a-b941-9a5fd74c9859 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:17.874775 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 445/887] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 19:00:17 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:17.875911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1966be-b25a-4e9a-b941-9a5fd74c9859 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:17.937926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:17.948414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f93b106-43b0-41ba-bd24-33c602b9b871 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:17.984094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362edf79-a3e4-4946-9d3c-c69dd16ed606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:17.984733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362edf79-a3e4-4946-9d3c-c69dd16ed606 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:17.985110 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362edf79-a3e4-4946-9d3c-c69dd16ed606 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:17.985473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362edf79-a3e4-4946-9d3c-c69dd16ed606 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:17.986068 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362edf79-a3e4-4946-9d3c-c69dd16ed606 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:17.987404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362edf79-a3e4-4946-9d3c-c69dd16ed606 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:17.992916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 443/888] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 19:00:17 2022] GET /identity/v3/services => generated 1268 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:00:17.994074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ef0a3-50b7-481e-ad84-38567aafdcc8 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:17.994352 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ef0a3-50b7-481e-ad84-38567aafdcc8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:17.994617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0ef0a3-50b7-481e-ad84-38567aafdcc8 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:17.997223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 444/889] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:18.001701 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51775772-f636-44e4-b5c6-4bfeef3ab064 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:18.002231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1966be-b25a-4e9a-b941-9a5fd74c9859 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:18.002461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51775772-f636-44e4-b5c6-4bfeef3ab064 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:18.002896 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51775772-f636-44e4-b5c6-4bfeef3ab064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:18.022812 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-51775772-f636-44e4-b5c6-4bfeef3ab064 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:18.024315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-51775772-f636-44e4-b5c6-4bfeef3ab064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:18.041745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ced4e5-fc89-4d49-b7a3-0991140bc6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:18.042364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ced4e5-fc89-4d49-b7a3-0991140bc6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:18.042659 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ced4e5-fc89-4d49-b7a3-0991140bc6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:18.043564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ced4e5-fc89-4d49-b7a3-0991140bc6d1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:18.044332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ced4e5-fc89-4d49-b7a3-0991140bc6d1 None admin] RBAC: Adding query filter params (interface=public, service_id=6c9aa92462904bff90d17a2a7a58c54c, region_id=RegionOne) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:18.044639 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ced4e5-fc89-4d49-b7a3-0991140bc6d1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:18.046252 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ced4e5-fc89-4d49-b7a3-0991140bc6d1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:18.056343 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 446/890] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 31 19:00:17 2022] GET /identity/v3/endpoints?service_id=6c9aa92462904bff90d17a2a7a58c54c&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:18.057603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ced4e5-fc89-4d49-b7a3-0991140bc6d1 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:18.112125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 445/891] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:18.130537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:18.182621 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ced4e5-fc89-4d49-b7a3-0991140bc6d1 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:18.227999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a28bf6-9ae2-4e39-af5a-daf3405e7ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:18.228652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a28bf6-9ae2-4e39-af5a-daf3405e7ade None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:18.228927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a28bf6-9ae2-4e39-af5a-daf3405e7ade None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:18.229268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a28bf6-9ae2-4e39-af5a-daf3405e7ade None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:18.229872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a28bf6-9ae2-4e39-af5a-daf3405e7ade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:18.232568 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a28bf6-9ae2-4e39-af5a-daf3405e7ade None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:18.238989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 447/892] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 19:00:18 2022] GET /identity/v3/services => generated 1268 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:00:18.259939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0e156-a550-461c-8b14-3a07bf5cd516 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:18.260295 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0e156-a550-461c-8b14-3a07bf5cd516 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:18.260493 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0e156-a550-461c-8b14-3a07bf5cd516 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:18.262305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 446/893] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:18.268010 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd0473-bf1a-445e-ae01-578ff5b0514b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:18.268308 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd0473-bf1a-445e-ae01-578ff5b0514b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:18.268572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fd0473-bf1a-445e-ae01-578ff5b0514b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:18.288883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-00fd0473-bf1a-445e-ae01-578ff5b0514b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:18.290092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-00fd0473-bf1a-445e-ae01-578ff5b0514b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:18.368584 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 448/894] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:18 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:18.402745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0e156-a550-461c-8b14-3a07bf5cd516 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:18.493348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:18.510999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0e156-a550-461c-8b14-3a07bf5cd516 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:18.551310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c4b0f-e64d-4192-9789-9539d3955864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:18.551310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c4b0f-e64d-4192-9789-9539d3955864 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:18.551310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c4b0f-e64d-4192-9789-9539d3955864 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:18.551310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c4b0f-e64d-4192-9789-9539d3955864 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:18.551686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c4b0f-e64d-4192-9789-9539d3955864 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:18.551686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c4b0f-e64d-4192-9789-9539d3955864 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-bf7c4b0f-e64d-4192-9789-9539d3955864 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:18.560291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:18.561321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:18.561321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:18.561321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:18.561321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:18.561321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:18.561321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:18.561321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:18.561321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:18.561321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:18.561321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:18.561321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:18.561321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:18.561321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:18.561321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:18.561321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:18.561321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:18.562306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:18.562306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:18.562306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:18.562306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:18.562306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:18.562306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:18.562306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:18.562306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:18.562306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:18.562306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:18.562306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:18.562306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:18.562306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:00:18.562306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:18.562306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 447/895] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 19:00:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:18.569571 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd0473-bf1a-445e-ae01-578ff5b0514b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:18.626751 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:18.641533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c4b0f-e64d-4192-9789-9539d3955864 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:18.691408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fd0473-bf1a-445e-ae01-578ff5b0514b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:18.738380 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ea7dc-d2d1-4983-af42-2a6cf0975a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:18.739356 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ea7dc-d2d1-4983-af42-2a6cf0975a8a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:18.739644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ea7dc-d2d1-4983-af42-2a6cf0975a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:18.739915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1ea7dc-d2d1-4983-af42-2a6cf0975a8a None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:18.740907 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ea7dc-d2d1-4983-af42-2a6cf0975a8a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:18.741669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ea7dc-d2d1-4983-af42-2a6cf0975a8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:18.743632 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1ea7dc-d2d1-4983-af42-2a6cf0975a8a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:18.751245 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c4b0f-e64d-4192-9789-9539d3955864 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:18.751488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 449/896] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 19:00:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:18.758169 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ea7dc-d2d1-4983-af42-2a6cf0975a8a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:18.798478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311df515-842a-4fb6-b521-4e1c3a3832c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:18.799463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311df515-842a-4fb6-b521-4e1c3a3832c6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:18.799987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311df515-842a-4fb6-b521-4e1c3a3832c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:18.800513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311df515-842a-4fb6-b521-4e1c3a3832c6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:18.812177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311df515-842a-4fb6-b521-4e1c3a3832c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:00:18.819024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-311df515-842a-4fb6-b521-4e1c3a3832c6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:00:18.819024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:18.819024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:18.819024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:18.819024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:18.819024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:18.819024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:18.819024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:18.819024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:18.819024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:18.819024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:18.819024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:18.819024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:00:18.819024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:00:18.819024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:00:18.819024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:00:18.819960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:00:18.819960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:00:18.819960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:00:18.819960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:00:18.819960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:18.819960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:18.819960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:18.819960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:18.819960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:18.819960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:18.819960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:18.819960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:18.819960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:18.819960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:18.819960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:18.819960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:18.820954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:18.820954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:18.820954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:18.820954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:18.820954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:00:18.820954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:00:18.820954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:00:18.820954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:00:18.820954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:00:18.820954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:00:18.820954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:00:18.820954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:18.820954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 448/897] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 31 19:00:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:18.830477 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311df515-842a-4fb6-b521-4e1c3a3832c6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:18.849038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:18.884077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1ea7dc-d2d1-4983-af42-2a6cf0975a8a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:18.898227 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:18.921753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2eb91-266c-4c6d-a932-f0ef016e6b46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:18.922616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2eb91-266c-4c6d-a932-f0ef016e6b46 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:18.923075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2eb91-266c-4c6d-a932-f0ef016e6b46 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:18.923552 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe2eb91-266c-4c6d-a932-f0ef016e6b46 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:18.924350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2eb91-266c-4c6d-a932-f0ef016e6b46 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:18.924825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2eb91-266c-4c6d-a932-f0ef016e6b46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:18.926402 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe2eb91-266c-4c6d-a932-f0ef016e6b46 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:18.937310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 450/898] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 19:00:18 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:18.957359 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311df515-842a-4fb6-b521-4e1c3a3832c6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:18.997151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb64e8-e798-4364-82e3-06900c36827b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:18.997809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb64e8-e798-4364-82e3-06900c36827b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:18.998058 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb64e8-e798-4364-82e3-06900c36827b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:18.998322 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fb64e8-e798-4364-82e3-06900c36827b None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:18.998995 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb64e8-e798-4364-82e3-06900c36827b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:18.999548 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb64e8-e798-4364-82e3-06900c36827b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:19.000920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fb64e8-e798-4364-82e3-06900c36827b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:19.010645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 449/899] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 31 19:00:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:19.016319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2eb91-266c-4c6d-a932-f0ef016e6b46 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:19.134717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe2eb91-266c-4c6d-a932-f0ef016e6b46 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:19.171375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0d579-a923-42d6-9a86-8ddcc4ed3fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:19.172005 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0d579-a923-42d6-9a86-8ddcc4ed3fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:19.172281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0d579-a923-42d6-9a86-8ddcc4ed3fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:19.172554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0d579-a923-42d6-9a86-8ddcc4ed3fe7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:19.183013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0d579-a923-42d6-9a86-8ddcc4ed3fe7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:19.184487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0d579-a923-42d6-9a86-8ddcc4ed3fe7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:19.202914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-e4a0d579-a923-42d6-9a86-8ddcc4ed3fe7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:19.202914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:19.202914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:19.202914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:19.202914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:19.202914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:19.202914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:19.202914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:19.202914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:19.202914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:19.202914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:19.202914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:19.202914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:19.202914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:19.202914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:19.202914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:19.203863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:19.205144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:19.205144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:19.205144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:19.205144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 451/900] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:19.210227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb64e8-e798-4364-82e3-06900c36827b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:19.288947 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:19.320535 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fb64e8-e798-4364-82e3-06900c36827b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:19.356283 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9061e9b-a4b4-4230-a726-77d0afefc300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:19.356907 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061e9b-a4b4-4230-a726-77d0afefc300 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:19.357182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061e9b-a4b4-4230-a726-77d0afefc300 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:19.357478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9061e9b-a4b4-4230-a726-77d0afefc300 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:19.358131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9061e9b-a4b4-4230-a726-77d0afefc300 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:19.358431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9061e9b-a4b4-4230-a726-77d0afefc300 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:19.359823 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9061e9b-a4b4-4230-a726-77d0afefc300 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:19.373543 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 450/901] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:19.379215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0d579-a923-42d6-9a86-8ddcc4ed3fe7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:19.412047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:19.486839 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0d579-a923-42d6-9a86-8ddcc4ed3fe7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:19.522741 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819aff78-3b40-4380-98ba-bbeff391aae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:19.523729 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819aff78-3b40-4380-98ba-bbeff391aae1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:19.524213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819aff78-3b40-4380-98ba-bbeff391aae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:19.524685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819aff78-3b40-4380-98ba-bbeff391aae1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:19.532625 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819aff78-3b40-4380-98ba-bbeff391aae1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:19.534356 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819aff78-3b40-4380-98ba-bbeff391aae1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:19.545807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-819aff78-3b40-4380-98ba-bbeff391aae1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:00:19.545807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:19.545807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:19.545807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:19.545807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:19.545807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:19.545807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:19.545807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:19.545807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:19.545807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:19.545807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:19.545807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:19.545807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:00:19.545807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:00:19.545807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:00:19.545807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:19.546732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:19.547684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:19.547684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:19.547684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:00:19.547684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:00:19.547684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:00:19.547684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:00:19.547684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:00:19.547684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:00:19.547684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:00:19.547684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:19.550164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 452/902] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 19:00:19 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:19.555788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9061e9b-a4b4-4230-a726-77d0afefc300 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:19.660555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9061e9b-a4b4-4230-a726-77d0afefc300 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:19.691135 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:19.696588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe81ce-e361-4491-a2b3-f51cac806c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:19.697215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe81ce-e361-4491-a2b3-f51cac806c54 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:19.697502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe81ce-e361-4491-a2b3-f51cac806c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:19.697770 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfe81ce-e361-4491-a2b3-f51cac806c54 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:19.698502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe81ce-e361-4491-a2b3-f51cac806c54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:19.698816 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe81ce-e361-4491-a2b3-f51cac806c54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:19.700842 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfe81ce-e361-4491-a2b3-f51cac806c54 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:19.718597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 451/903] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 31 19:00:19 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:19.723835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819aff78-3b40-4380-98ba-bbeff391aae1 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:19.820534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:19.833744 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819aff78-3b40-4380-98ba-bbeff391aae1 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:19.869253 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea4b3cd-4fea-4640-840d-aa5e41b1038f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:19.870251 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4b3cd-4fea-4640-840d-aa5e41b1038f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:19.870776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4b3cd-4fea-4640-840d-aa5e41b1038f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:19.871825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea4b3cd-4fea-4640-840d-aa5e41b1038f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:19.948850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4b3cd-4fea-4640-840d-aa5e41b1038f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:19.950310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea4b3cd-4fea-4640-840d-aa5e41b1038f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:20.042504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:20.058332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 453/904] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:00:19 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:00:20.064153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe81ce-e361-4491-a2b3-f51cac806c54 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:20.109417 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b539b3-28b1-4383-bfc8-b4388da8e773 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:20.110021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b539b3-28b1-4383-bfc8-b4388da8e773 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:20.110520 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b539b3-28b1-4383-bfc8-b4388da8e773 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:20.112417 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 454/905] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:20.118843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15eb37-e8f0-4cc7-82a4-a530beedebf1 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:20.119481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15eb37-e8f0-4cc7-82a4-a530beedebf1 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:20.119957 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f15eb37-e8f0-4cc7-82a4-a530beedebf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:20.139883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4f15eb37-e8f0-4cc7-82a4-a530beedebf1 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:20.141477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4f15eb37-e8f0-4cc7-82a4-a530beedebf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:20.172164 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfe81ce-e361-4491-a2b3-f51cac806c54 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:20.211884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f552190-c4d2-4880-83c7-3fa2a560101c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:20.212806 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f552190-c4d2-4880-83c7-3fa2a560101c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:20.213321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f552190-c4d2-4880-83c7-3fa2a560101c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:20.216559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f552190-c4d2-4880-83c7-3fa2a560101c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:20.229308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f552190-c4d2-4880-83c7-3fa2a560101c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:20.231702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f552190-c4d2-4880-83c7-3fa2a560101c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:20.239563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 455/906] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:00:20.241405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f101d-ce18-4839-b6e2-86d2504864c7 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:20.241695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f101d-ce18-4839-b6e2-86d2504864c7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:20.241975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f101d-ce18-4839-b6e2-86d2504864c7 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:20.243964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 456/907] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:20.249986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-2f552190-c4d2-4880-83c7-3fa2a560101c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:20.249986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:20.249986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:20.249986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:20.249986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:20.249986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:20.249986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:20.249986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:20.249986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:20.249986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:20.249986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:20.249986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:20.249986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:20.249986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:20.249986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:20.249986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:20.250895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:20.251751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:20.251751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:20.251751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:20.251924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e08c50-d94a-4612-98c8-1a2a369d7071 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:20.251924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e08c50-d94a-4612-98c8-1a2a369d7071 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:20.251924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e08c50-d94a-4612-98c8-1a2a369d7071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:20.253524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 452/908] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:20.259060 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f552190-c4d2-4880-83c7-3fa2a560101c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:20.274644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a0e08c50-d94a-4612-98c8-1a2a369d7071 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:20.276073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a0e08c50-d94a-4612-98c8-1a2a369d7071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:20.316348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:20.347070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 457/909] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:20 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:20.383735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f552190-c4d2-4880-83c7-3fa2a560101c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:20.419031 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b68be7-6898-4557-abf3-da9015c4c5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:20.419637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b68be7-6898-4557-abf3-da9015c4c5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:20.419916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b68be7-6898-4557-abf3-da9015c4c5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:20.420193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b68be7-6898-4557-abf3-da9015c4c5c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:20.420872 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b68be7-6898-4557-abf3-da9015c4c5c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:20.421174 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b68be7-6898-4557-abf3-da9015c4c5c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:20.422551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b68be7-6898-4557-abf3-da9015c4c5c2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:20.434768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 453/910] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:20.440309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e08c50-d94a-4612-98c8-1a2a369d7071 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:20.494120 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:20.530179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b68be7-6898-4557-abf3-da9015c4c5c2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:20.551967 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e08c50-d94a-4612-98c8-1a2a369d7071 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:20.593815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b40f3-6510-4382-8a05-227793a81729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:20.594445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b40f3-6510-4382-8a05-227793a81729 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:20.594726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b40f3-6510-4382-8a05-227793a81729 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:20.594999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7b40f3-6510-4382-8a05-227793a81729 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:20.604210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b40f3-6510-4382-8a05-227793a81729 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:20.605888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7b40f3-6510-4382-8a05-227793a81729 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:20.619878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-ea7b40f3-6510-4382-8a05-227793a81729 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:20.619878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:20.619878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:20.619878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:20.619878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:20.619878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:20.619878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:20.619878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:20.619878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:20.619878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:20.619878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:20.619878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:20.619878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:00:20.619878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:00:20.619878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:20.620812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:20.621783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:00:20.621783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:00:20.621783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:00:20.621783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:00:20.621783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:00:20.621783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:00:20.621783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:20.621783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:20.622854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 458/911] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:20.623823 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b40f3-6510-4382-8a05-227793a81729 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:20.651294 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b68be7-6898-4557-abf3-da9015c4c5c2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:20.653579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:20.688392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743c452-e6b0-4421-9cdb-d109984156bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:20.689011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743c452-e6b0-4421-9cdb-d109984156bc None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:20.689321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743c452-e6b0-4421-9cdb-d109984156bc None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:20.689592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743c452-e6b0-4421-9cdb-d109984156bc None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:20.690162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743c452-e6b0-4421-9cdb-d109984156bc None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:20.691513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743c452-e6b0-4421-9cdb-d109984156bc None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-c743c452-e6b0-4421-9cdb-d109984156bc None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:20.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:20.702293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:20.702293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:20.702293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:20.702293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:20.702293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:20.702293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:20.702293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:20.702293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:20.702293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:20.702293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:20.702293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:20.702293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:20.702293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:20.702293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:20.702293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:20.702293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:20.706670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:20.706670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:20.706670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:20.706670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:20.706670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:20.706670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:20.706670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:20.706670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:20.706670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:20.706670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:20.706670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:20.706670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:20.706670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 31 19:00:20.706670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:20.706670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 454/912] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 19:00:20 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:20.707819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743c452-e6b0-4421-9cdb-d109984156bc None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:20.732560 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7b40f3-6510-4382-8a05-227793a81729 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:20.741835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:20.769455 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b5039-1694-446f-b15e-539b74dfb02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:20.770112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b5039-1694-446f-b15e-539b74dfb02b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:20.770394 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b5039-1694-446f-b15e-539b74dfb02b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:20.770667 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39b5039-1694-446f-b15e-539b74dfb02b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:20.771249 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b5039-1694-446f-b15e-539b74dfb02b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:20.772607 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39b5039-1694-446f-b15e-539b74dfb02b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:20.782203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-b39b5039-1694-446f-b15e-539b74dfb02b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:00:20.782203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:20.782203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:20.782203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:20.782203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:20.782203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:20.782203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:20.782203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:20.782203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:00:20.782203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:20.782203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:20.782203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:20.782203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:20.782203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:20.782203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:20.782203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:20.783144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:20.784022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:20.784022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:20.784022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:20.784022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:20.784022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:20.784022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:20.784022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:20.784022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:20.784022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:20.784022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:20.784022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:20.784022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:20.784022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 31 19:00:20.784022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:20.784022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 459/913] 10.222.0.65 () {56 vars in 1132 bytes} [Wed Aug 31 19:00:20 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:20.784821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b5039-1694-446f-b15e-539b74dfb02b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:20.827993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743c452-e6b0-4421-9cdb-d109984156bc None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:20.864371 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7b1c0-d834-413d-85b1-a4dd0ef56fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:20.865081 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7b1c0-d834-413d-85b1-a4dd0ef56fca None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:20.865382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7b1c0-d834-413d-85b1-a4dd0ef56fca None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:20.865654 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c7b1c0-d834-413d-85b1-a4dd0ef56fca None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:20.866456 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7b1c0-d834-413d-85b1-a4dd0ef56fca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:20.866756 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7b1c0-d834-413d-85b1-a4dd0ef56fca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:20.868237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c7b1c0-d834-413d-85b1-a4dd0ef56fca None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:20.886352 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 455/914] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:00:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:20.886352 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7b1c0-d834-413d-85b1-a4dd0ef56fca None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:20.893492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39b5039-1694-446f-b15e-539b74dfb02b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:20.912489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:20.930035 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d484c-2974-4ea8-879f-5a49d4d84afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:20.930639 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d484c-2974-4ea8-879f-5a49d4d84afa None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:20.930914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d484c-2974-4ea8-879f-5a49d4d84afa None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:20.931185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d484c-2974-4ea8-879f-5a49d4d84afa None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:20.931838 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d484c-2974-4ea8-879f-5a49d4d84afa None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:20.932136 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d484c-2974-4ea8-879f-5a49d4d84afa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:20.933592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d484c-2974-4ea8-879f-5a49d4d84afa None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:20.942783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 460/915] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 19:00:20 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:20.942783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d484c-2974-4ea8-879f-5a49d4d84afa None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:20.978768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:20.994864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c7b1c0-d834-413d-85b1-a4dd0ef56fca None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:21.029433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68b149-8042-4478-b0f9-0a932862205f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:21.029967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68b149-8042-4478-b0f9-0a932862205f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:21.030245 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68b149-8042-4478-b0f9-0a932862205f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:21.030516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68b149-8042-4478-b0f9-0a932862205f None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:21.031136 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68b149-8042-4478-b0f9-0a932862205f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:21.031433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68b149-8042-4478-b0f9-0a932862205f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:21.032781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68b149-8042-4478-b0f9-0a932862205f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:21.042009 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 456/916] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 19:00:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:21.042009 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68b149-8042-4478-b0f9-0a932862205f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:21.049802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d484c-2974-4ea8-879f-5a49d4d84afa None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:21.084798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894dd91-e9b3-402a-93d8-055fe848265c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:21.085842 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894dd91-e9b3-402a-93d8-055fe848265c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:21.086323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894dd91-e9b3-402a-93d8-055fe848265c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:21.086799 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894dd91-e9b3-402a-93d8-055fe848265c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:21.088123 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894dd91-e9b3-402a-93d8-055fe848265c None admin] RBAC: Adding query filter params (role.id=f37b082e86f54a6a936cf9a83fe956a7, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=dd730375a996433d982b26c8339ec4ee) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:21.088653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894dd91-e9b3-402a-93d8-055fe848265c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:21.091343 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894dd91-e9b3-402a-93d8-055fe848265c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:21.099255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 461/917] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:00:20 2022] GET /identity/v3/role_assignments?user.id=dd730375a996433d982b26c8339ec4ee&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=f37b082e86f54a6a936cf9a83fe956a7 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:21.100359 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c7c7f-628b-4a03-802d-7150828b8977 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:21.100631 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c7c7f-628b-4a03-802d-7150828b8977 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:21.100892 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265c7c7f-628b-4a03-802d-7150828b8977 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:21.103972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 462/918] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:21 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:21.103972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c7c7f-628b-4a03-802d-7150828b8977 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:21.133900 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:21.146414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68b149-8042-4478-b0f9-0a932862205f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:21.184158 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99777d-7b37-432a-9ea8-79c5ba95f175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:21.184775 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99777d-7b37-432a-9ea8-79c5ba95f175 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:21.185051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99777d-7b37-432a-9ea8-79c5ba95f175 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:21.185346 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99777d-7b37-432a-9ea8-79c5ba95f175 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:21.185975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99777d-7b37-432a-9ea8-79c5ba95f175 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:21.186262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99777d-7b37-432a-9ea8-79c5ba95f175 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:21.187641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d99777d-7b37-432a-9ea8-79c5ba95f175 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:21.203703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 457/919] 10.222.0.65 () {56 vars in 1114 bytes} [Wed Aug 31 19:00:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:21.203703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8822c86-46b3-49b1-93ef-5ee9c4dfc021 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:21.203703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8822c86-46b3-49b1-93ef-5ee9c4dfc021 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:21.203703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8822c86-46b3-49b1-93ef-5ee9c4dfc021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:21.225445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f8822c86-46b3-49b1-93ef-5ee9c4dfc021 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:21.226403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:21.226824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f8822c86-46b3-49b1-93ef-5ee9c4dfc021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:21.230900 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265c7c7f-628b-4a03-802d-7150828b8977 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:21.270256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64670842-e443-4098-ba35-5731fd216cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:21.270884 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64670842-e443-4098-ba35-5731fd216cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:21.271218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64670842-e443-4098-ba35-5731fd216cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:21.271398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64670842-e443-4098-ba35-5731fd216cc1 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:21.272026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64670842-e443-4098-ba35-5731fd216cc1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:21.272329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64670842-e443-4098-ba35-5731fd216cc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:21.273727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64670842-e443-4098-ba35-5731fd216cc1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:21.283831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 463/920] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 19:00:21 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:21.284565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64670842-e443-4098-ba35-5731fd216cc1 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:21.309281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 458/921] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:21 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:21.310929 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8822c86-46b3-49b1-93ef-5ee9c4dfc021 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:21.402467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64670842-e443-4098-ba35-5731fd216cc1 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:21.408619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:21.427579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8822c86-46b3-49b1-93ef-5ee9c4dfc021 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:21.439270 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81840a8b-ab51-495d-a6ed-647037c774c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:21.440173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81840a8b-ab51-495d-a6ed-647037c774c7 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:21.440644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81840a8b-ab51-495d-a6ed-647037c774c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:21.441131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81840a8b-ab51-495d-a6ed-647037c774c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:21.442027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81840a8b-ab51-495d-a6ed-647037c774c7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:21.443587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81840a8b-ab51-495d-a6ed-647037c774c7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:21.470800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 464/922] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:00:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:00:21.483344 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff179b-471e-4de7-9100-5a046a664921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:21.483979 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff179b-471e-4de7-9100-5a046a664921 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:21.484266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff179b-471e-4de7-9100-5a046a664921 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:21.484538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff179b-471e-4de7-9100-5a046a664921 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:21.485328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff179b-471e-4de7-9100-5a046a664921 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:21.486943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff179b-471e-4de7-9100-5a046a664921 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:21.514347 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 459/923] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:00:21 2022] POST /identity/v3/endpoints => generated 347 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:00:21.589034 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81840a8b-ab51-495d-a6ed-647037c774c7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:21.616823 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:21.704157 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81840a8b-ab51-495d-a6ed-647037c774c7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:21.753859 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a851b38-d4e1-4cf1-9e5f-ff41eb205fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:21.754468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a851b38-d4e1-4cf1-9e5f-ff41eb205fca None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:21.754741 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a851b38-d4e1-4cf1-9e5f-ff41eb205fca None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:21.755006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a851b38-d4e1-4cf1-9e5f-ff41eb205fca None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:21.755639 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a851b38-d4e1-4cf1-9e5f-ff41eb205fca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:21.756987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a851b38-d4e1-4cf1-9e5f-ff41eb205fca None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:21.776144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 465/924] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 19:00:21 2022] POST /identity/v3/services => generated 241 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:00:23.046163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654a7ca-4fba-4523-8c7d-4506a28cc04d None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:23.046688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654a7ca-4fba-4523-8c7d-4506a28cc04d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:23.046688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9654a7ca-4fba-4523-8c7d-4506a28cc04d None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:23.048295 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 460/925] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:23.053460 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828532bf-6c67-4d4a-bb8d-75cc076b5340 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:23.053880 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828532bf-6c67-4d4a-bb8d-75cc076b5340 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:23.054046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828532bf-6c67-4d4a-bb8d-75cc076b5340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:23.072382 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-828532bf-6c67-4d4a-bb8d-75cc076b5340 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:23.073610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-828532bf-6c67-4d4a-bb8d-75cc076b5340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:23.153158 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 466/926] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:23.432020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654a7ca-4fba-4523-8c7d-4506a28cc04d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:23.492026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e83d4-2294-434c-ac3f-aa0c3c007c16 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:23.492679 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e83d4-2294-434c-ac3f-aa0c3c007c16 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:23.493154 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e83d4-2294-434c-ac3f-aa0c3c007c16 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:23.494956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 467/927] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:23.500874 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c7155-f889-4e09-933d-0d161959bb79 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:23.501401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c7155-f889-4e09-933d-0d161959bb79 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:23.501874 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65c7155-f889-4e09-933d-0d161959bb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:23.508592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:23.524274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-d65c7155-f889-4e09-933d-0d161959bb79 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:23.525569 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d65c7155-f889-4e09-933d-0d161959bb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:23.543468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9654a7ca-4fba-4523-8c7d-4506a28cc04d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:23.578378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b531b8b-3fb0-431a-abf5-6fb114668fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:23.579004 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b531b8b-3fb0-431a-abf5-6fb114668fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:23.579280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b531b8b-3fb0-431a-abf5-6fb114668fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:23.579548 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b531b8b-3fb0-431a-abf5-6fb114668fa0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:23.582605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:23.592307 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b531b8b-3fb0-431a-abf5-6fb114668fa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:00:23.597523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-0b531b8b-3fb0-431a-abf5-6fb114668fa0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:00:23.597523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:23.597523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:23.597523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:23.597523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:23.597523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:23.597523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:23.597523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:23.597523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:23.597523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:23.597523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:23.597523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:23.597523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:00:23.597523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:00:23.597523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:00:23.597523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:00:23.598450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:00:23.598450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:00:23.598450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:00:23.598450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:00:23.598450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:23.598450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:23.598450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:23.598450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:23.598450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:23.598450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:23.598450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:23.598450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:23.598450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:23.598450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:23.598450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:23.598450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:23.599850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:23.599850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:23.599850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:23.599850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:23.599850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:00:23.599850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:00:23.599850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:00:23.599850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:00:23.599850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:00:23.599850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:00:23.599850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:00:23.599850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:23.600518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 468/928] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:00:23.600518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6488000-ffc1-40cd-a23f-95b214121454 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:23.600518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6488000-ffc1-40cd-a23f-95b214121454 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:23.600518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6488000-ffc1-40cd-a23f-95b214121454 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:23.601850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 469/929] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:23.606437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 461/930] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 31 19:00:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:23.607946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb3b51-6e8c-4326-97a6-09dd95cd1f05 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:23.608586 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb3b51-6e8c-4326-97a6-09dd95cd1f05 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:23.609220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addb3b51-6e8c-4326-97a6-09dd95cd1f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:23.611761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b531b8b-3fb0-431a-abf5-6fb114668fa0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:23.630788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-addb3b51-6e8c-4326-97a6-09dd95cd1f05 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:23.631972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-addb3b51-6e8c-4326-97a6-09dd95cd1f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:23.713287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 470/931] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:23.723449 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b531b8b-3fb0-431a-abf5-6fb114668fa0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:23.737853 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b01261-f1f6-4a29-b753-7649d8df8b6f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:23.738126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b01261-f1f6-4a29-b753-7649d8df8b6f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:23.738405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b01261-f1f6-4a29-b753-7649d8df8b6f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:23.741330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 471/932] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:00:23.744545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274573c4-4ec3-4e24-9410-b9945ccee5c6 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:23.744827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274573c4-4ec3-4e24-9410-b9945ccee5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:23.745538 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274573c4-4ec3-4e24-9410-b9945ccee5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:23.763667 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b7c31-7d00-4df6-959a-a9415758f87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:23.764618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b7c31-7d00-4df6-959a-a9415758f87f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:23.765153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b7c31-7d00-4df6-959a-a9415758f87f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:23.765655 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b7c31-7d00-4df6-959a-a9415758f87f None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:23.766515 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b7c31-7d00-4df6-959a-a9415758f87f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:23.767059 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b7c31-7d00-4df6-959a-a9415758f87f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:23.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-274573c4-4ec3-4e24-9410-b9945ccee5c6 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:23.768893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-274573c4-4ec3-4e24-9410-b9945ccee5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:23.769133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b7c31-7d00-4df6-959a-a9415758f87f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:23.778411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 462/933] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 31 19:00:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:23.785686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b7c31-7d00-4df6-959a-a9415758f87f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:23.831648 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:23.846246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:23.849464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 472/934] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:23.868911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274573c4-4ec3-4e24-9410-b9945ccee5c6 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:23.896209 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b7c31-7d00-4df6-959a-a9415758f87f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:23.932687 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5fbd1-1cc6-440b-9e82-29389a5f1c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:23.933362 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5fbd1-1cc6-440b-9e82-29389a5f1c3f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:23.934192 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5fbd1-1cc6-440b-9e82-29389a5f1c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:23.934552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5fbd1-1cc6-440b-9e82-29389a5f1c3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:23.946919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5fbd1-1cc6-440b-9e82-29389a5f1c3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:23.948331 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5fbd1-1cc6-440b-9e82-29389a5f1c3f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:23.963631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-41c5fbd1-1cc6-440b-9e82-29389a5f1c3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:23.963631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:23.963631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:23.963631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:23.963631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:23.963631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:23.963631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:23.963631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:23.963631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:23.963631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:23.963631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:23.963631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:23.963631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:23.963631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:23.963631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:23.963631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:23.964530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:23.966044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:23.966044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:23.966044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:23.969541 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 463/935] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:23.971273 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5fbd1-1cc6-440b-9e82-29389a5f1c3f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:24.004611 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274573c4-4ec3-4e24-9410-b9945ccee5c6 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:24.045378 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537788b-9a9b-4654-b1f8-86f7df56a81f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:24.046001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537788b-9a9b-4654-b1f8-86f7df56a81f None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:24.046271 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537788b-9a9b-4654-b1f8-86f7df56a81f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:24.046522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537788b-9a9b-4654-b1f8-86f7df56a81f None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:24.047169 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d537788b-9a9b-4654-b1f8-86f7df56a81f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:24.048868 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d537788b-9a9b-4654-b1f8-86f7df56a81f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:24.051191 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: WARNING keystone.server.flask.common [None req-d537788b-9a9b-4654-b1f8-86f7df56a81f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 19:00:24.069313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.sql.core [None req-d537788b-9a9b-4654-b1f8-86f7df56a81f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 19:00:24.069313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 19:00:24.069313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [parameters: {'id': '265a555a2eb54471b57aa27b9b2e445e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 19:00:24.069313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81627) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 19:00:24.076757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-d537788b-9a9b-4654-b1f8-86f7df56a81f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 265a555a2eb54471b57aa27b9b2e445e.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 265a555a2eb54471b57aa27b9b2e445e. Aug 31 19:00:24.076757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:24.076757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 19:00:24.076757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 19:00:24.076757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 19:00:24.076757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 19:00:24.076757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 19:00:24.076757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 19:00:24.076757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 19:00:24.076757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application conn.query(q) Aug 31 19:00:24.076757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 19:00:24.076757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 19:00:24.076757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application result.read() Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 31 19:00:24.077718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application next(self.gen) Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application yield resource Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application next(self.gen) Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise self.value Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application session.commit() Aug 31 19:00:24.083513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.session.flush() Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application compat.raise_( Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 19:00:24.084398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 19:00:24.085354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 19:00:24.085354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 19:00:24.085354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 19:00:24.085354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 19:00:24.085354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 31 19:00:24.085354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 31 19:00:24.085354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 19:00:24.085354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 19:00:24.085354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 19:00:24.085354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 19:00:24.085354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 19:00:24.085354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 19:00:24.085354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 19:00:24.085354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 19:00:24.085354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application conn.query(q) Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application result.read() Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 19:00:24.086152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 19:00:24.087048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 19:00:24.087048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 19:00:24.087048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 19:00:24.087048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 19:00:24.087048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 19:00:24.087048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 19:00:24.087048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 31 19:00:24.087048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 31 19:00:24.087048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [parameters: {'id': '265a555a2eb54471b57aa27b9b2e445e', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 31 19:00:24.087048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 19:00:24.087048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:24.087048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:24.087048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:24.087048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:24.087048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:24.087829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:24.088739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:24.088739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 31 19:00:24.088739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 31 19:00:24.088739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:24.088739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:24.088739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 31 19:00:24.088739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 31 19:00:24.088739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 265a555a2eb54471b57aa27b9b2e445e. Aug 31 19:00:24.088739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:24.088739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 473/936] 10.222.0.65 () {60 vars in 1140 bytes} [Wed Aug 31 19:00:23 2022] POST /identity/v3/projects => generated 262 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:24.088739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537788b-9a9b-4654-b1f8-86f7df56a81f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:24.090141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5fbd1-1cc6-440b-9e82-29389a5f1c3f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:24.098911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:24.129417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038b216-2d51-4ec8-90f6-afcb5a2ed461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:24.130021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038b216-2d51-4ec8-90f6-afcb5a2ed461 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:24.130294 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038b216-2d51-4ec8-90f6-afcb5a2ed461 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:24.130560 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f038b216-2d51-4ec8-90f6-afcb5a2ed461 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:24.131214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038b216-2d51-4ec8-90f6-afcb5a2ed461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:24.131494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038b216-2d51-4ec8-90f6-afcb5a2ed461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:24.132839 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f038b216-2d51-4ec8-90f6-afcb5a2ed461 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:24.141479 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:24.145267 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 464/937] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:24.146093 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038b216-2d51-4ec8-90f6-afcb5a2ed461 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:24.196611 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537788b-9a9b-4654-b1f8-86f7df56a81f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:24.232695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc8518-2598-4d94-9136-64fe22f6c0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:24.233592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc8518-2598-4d94-9136-64fe22f6c0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:24.233592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc8518-2598-4d94-9136-64fe22f6c0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:24.233806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcc8518-2598-4d94-9136-64fe22f6c0b0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:24.234370 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc8518-2598-4d94-9136-64fe22f6c0b0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:24.235923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcc8518-2598-4d94-9136-64fe22f6c0b0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-1bcc8518-2598-4d94-9136-64fe22f6c0b0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:24.244808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:24.246095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:24.246095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:24.246095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:24.246095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:24.246095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:24.246095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:24.246095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:24.246095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:24.246095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:24.246095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:24.246095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:24.246095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:24.246095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:24.246095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:24.246095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:24.246095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:24.247409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:24.247409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:24.247409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:24.247409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:24.247409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:24.247409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:24.247409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:24.247409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:24.247409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:24.247409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:24.247409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:24.247409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:24.247409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:00:24.247409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:24.248263 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f038b216-2d51-4ec8-90f6-afcb5a2ed461 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:24.248338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 474/938] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:24 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:24.248338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc8518-2598-4d94-9136-64fe22f6c0b0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:24.281415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f629d3b-e596-40e1-85b3-611dbea91b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:24.282036 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f629d3b-e596-40e1-85b3-611dbea91b23 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:24.282313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f629d3b-e596-40e1-85b3-611dbea91b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:24.282580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f629d3b-e596-40e1-85b3-611dbea91b23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:24.289090 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f629d3b-e596-40e1-85b3-611dbea91b23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:24.290521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f629d3b-e596-40e1-85b3-611dbea91b23 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:24.302054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-3f629d3b-e596-40e1-85b3-611dbea91b23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:24.302054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:24.302054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:24.302054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:24.302054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:24.302054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:24.302054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:24.302054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:24.302054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:24.302054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:24.302054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:24.302054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:24.302054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:00:24.302054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:00:24.302054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:24.302934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:24.303925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:00:24.303925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:00:24.303925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:00:24.303925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:00:24.303925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:00:24.303925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:00:24.303925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:24.303925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:24.303925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 465/939] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:24.304419 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f629d3b-e596-40e1-85b3-611dbea91b23 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:24.333416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:24.352897 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcc8518-2598-4d94-9136-64fe22f6c0b0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:24.365421 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:24.386413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2398df-9ab8-4d72-b4b1-757a25d624d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:24.387562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2398df-9ab8-4d72-b4b1-757a25d624d1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:24.387836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2398df-9ab8-4d72-b4b1-757a25d624d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:24.388105 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2398df-9ab8-4d72-b4b1-757a25d624d1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:24.388680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2398df-9ab8-4d72-b4b1-757a25d624d1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:24.390071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2398df-9ab8-4d72-b4b1-757a25d624d1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-4f2398df-9ab8-4d72-b4b1-757a25d624d1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:24.398664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:24.399815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:24.399815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:24.399815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:24.399815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:24.399815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:24.399815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:24.399815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:24.399815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:24.399815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:24.399815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:24.399815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:24.399815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:24.399815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:24.399815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:24.399815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:24.399815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:24.400670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:24.400670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:24.400670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:24.400670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:24.400670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:24.400670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:24.400670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:24.400670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:24.400670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:24.400670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:24.400670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:24.400670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:24.400670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:00:24.400670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:24.400670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 475/940] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 19:00:24 2022] GET /identity/v3/services/object-store => generated 93 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:24.402047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2398df-9ab8-4d72-b4b1-757a25d624d1 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:24.402228 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f629d3b-e596-40e1-85b3-611dbea91b23 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:24.435657 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e07484-ab6a-435a-87b4-9dd38b1475bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:24.436280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e07484-ab6a-435a-87b4-9dd38b1475bb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:24.436559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e07484-ab6a-435a-87b4-9dd38b1475bb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:24.436831 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e07484-ab6a-435a-87b4-9dd38b1475bb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:24.444202 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e07484-ab6a-435a-87b4-9dd38b1475bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:24.445661 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e07484-ab6a-435a-87b4-9dd38b1475bb None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:24.457052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-f1e07484-ab6a-435a-87b4-9dd38b1475bb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:00:24.457052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:24.457052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:24.457052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:24.457052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:24.457052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:24.457052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:24.457052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:24.457052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:24.457052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:24.457052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:24.457052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:24.457052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:00:24.457052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:00:24.457052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:00:24.457052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:24.458123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:24.459096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:24.459096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:24.459096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:00:24.459096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:00:24.459096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:00:24.459096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:00:24.459096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:00:24.459096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:00:24.459096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:00:24.459096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:24.459096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 466/941] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 19:00:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:24.459096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e07484-ab6a-435a-87b4-9dd38b1475bb None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:24.506472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2398df-9ab8-4d72-b4b1-757a25d624d1 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:24.541207 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df556e-a81b-4bbf-8a92-97350c533ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:24.541816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df556e-a81b-4bbf-8a92-97350c533ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:24.542093 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df556e-a81b-4bbf-8a92-97350c533ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:24.542363 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9df556e-a81b-4bbf-8a92-97350c533ba1 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:24.543022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df556e-a81b-4bbf-8a92-97350c533ba1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:24.543318 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df556e-a81b-4bbf-8a92-97350c533ba1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:24.544675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9df556e-a81b-4bbf-8a92-97350c533ba1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:24.548329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:24.556720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 476/942] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 19:00:24 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:24.556720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df556e-a81b-4bbf-8a92-97350c533ba1 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:24.559139 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e07484-ab6a-435a-87b4-9dd38b1475bb None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:24.593102 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718385f-a325-4881-b702-9d6eaafc2071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:24.593722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718385f-a325-4881-b702-9d6eaafc2071 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:24.594002 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718385f-a325-4881-b702-9d6eaafc2071 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:24.594422 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1718385f-a325-4881-b702-9d6eaafc2071 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:24.594538 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:24.594926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718385f-a325-4881-b702-9d6eaafc2071 None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:24.595227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718385f-a325-4881-b702-9d6eaafc2071 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:24.596654 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1718385f-a325-4881-b702-9d6eaafc2071 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:24.614102 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 467/943] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 19:00:24 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:24.614967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718385f-a325-4881-b702-9d6eaafc2071 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:24.660741 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9df556e-a81b-4bbf-8a92-97350c533ba1 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:24.699510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244b0b2-ff97-46d3-9a34-b1020f4dcb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:24.700139 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244b0b2-ff97-46d3-9a34-b1020f4dcb5a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:24.700415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244b0b2-ff97-46d3-9a34-b1020f4dcb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:24.700687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244b0b2-ff97-46d3-9a34-b1020f4dcb5a None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:24.701350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a244b0b2-ff97-46d3-9a34-b1020f4dcb5a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:24.701654 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a244b0b2-ff97-46d3-9a34-b1020f4dcb5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:24.703039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a244b0b2-ff97-46d3-9a34-b1020f4dcb5a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:24.713179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 477/944] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 19:00:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:24.713179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244b0b2-ff97-46d3-9a34-b1020f4dcb5a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:24.738813 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1718385f-a325-4881-b702-9d6eaafc2071 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:24.780773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f265061-0d56-40ed-9453-bcfa76bfca76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:24.781693 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f265061-0d56-40ed-9453-bcfa76bfca76 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:24.781974 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f265061-0d56-40ed-9453-bcfa76bfca76 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:24.782249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f265061-0d56-40ed-9453-bcfa76bfca76 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:24.782979 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f265061-0d56-40ed-9453-bcfa76bfca76 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:24.783279 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f265061-0d56-40ed-9453-bcfa76bfca76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:24.784706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f265061-0d56-40ed-9453-bcfa76bfca76 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:24.802330 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 468/945] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 31 19:00:24 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:24.803467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f265061-0d56-40ed-9453-bcfa76bfca76 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:24.815866 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:24.824862 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a244b0b2-ff97-46d3-9a34-b1020f4dcb5a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:24.827582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:24.862955 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f79a19-e5e1-4883-9848-9e59ebc93b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:24.863564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f79a19-e5e1-4883-9848-9e59ebc93b0c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:24.863845 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f79a19-e5e1-4883-9848-9e59ebc93b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:24.864118 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f79a19-e5e1-4883-9848-9e59ebc93b0c None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:24.864748 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f79a19-e5e1-4883-9848-9e59ebc93b0c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:24.865048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f79a19-e5e1-4883-9848-9e59ebc93b0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:24.866437 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f79a19-e5e1-4883-9848-9e59ebc93b0c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:24.877213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 478/946] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 19:00:24 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:24.877213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f79a19-e5e1-4883-9848-9e59ebc93b0c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:24.919193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f265061-0d56-40ed-9453-bcfa76bfca76 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:24.959457 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120c3ea-9849-4e37-8b58-af5bc7bf2e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:24.960077 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120c3ea-9849-4e37-8b58-af5bc7bf2e59 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:24.960353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120c3ea-9849-4e37-8b58-af5bc7bf2e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:24.960620 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9120c3ea-9849-4e37-8b58-af5bc7bf2e59 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:24.961390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120c3ea-9849-4e37-8b58-af5bc7bf2e59 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:24.961684 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120c3ea-9849-4e37-8b58-af5bc7bf2e59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:24.963121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9120c3ea-9849-4e37-8b58-af5bc7bf2e59 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:24.969225 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 469/947] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 19:00:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:24.975290 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120c3ea-9849-4e37-8b58-af5bc7bf2e59 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:24.990996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f79a19-e5e1-4883-9848-9e59ebc93b0c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:25.031439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9c1d3-4ad4-4362-9f8b-97b90804a419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:25.032530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9c1d3-4ad4-4362-9f8b-97b90804a419 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:25.033070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9c1d3-4ad4-4362-9f8b-97b90804a419 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:25.033602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a9c1d3-4ad4-4362-9f8b-97b90804a419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:25.046420 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9c1d3-4ad4-4362-9f8b-97b90804a419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:25.048346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a9c1d3-4ad4-4362-9f8b-97b90804a419 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:25.057560 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:25.067165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-60a9c1d3-4ad4-4362-9f8b-97b90804a419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:25.067165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:25.067165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:25.067165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:25.067165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:25.067165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:25.067165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:25.067165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:25.067165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:25.067165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:25.067165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:25.067165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:25.067165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:25.067165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:25.067165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:25.067165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:25.068178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:25.069086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:25.069086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:25.069086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:25.069086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 479/948] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:25.077094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9c1d3-4ad4-4362-9f8b-97b90804a419 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:25.095534 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9120c3ea-9849-4e37-8b58-af5bc7bf2e59 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:25.101566 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:25.136837 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71251ab4-c91f-43b6-8583-a66dc165782e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:25.137826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71251ab4-c91f-43b6-8583-a66dc165782e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:25.138295 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71251ab4-c91f-43b6-8583-a66dc165782e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:25.138732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71251ab4-c91f-43b6-8583-a66dc165782e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:25.139591 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71251ab4-c91f-43b6-8583-a66dc165782e None admin] RBAC: Adding query filter params (interface=public, service_id=8b8f7950ecc3420e900acee60578266c, region_id=RegionOne) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:25.140065 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71251ab4-c91f-43b6-8583-a66dc165782e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:25.141629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71251ab4-c91f-43b6-8583-a66dc165782e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:25.150376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 470/949] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 31 19:00:24 2022] GET /identity/v3/endpoints?service_id=8b8f7950ecc3420e900acee60578266c&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:25.159145 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71251ab4-c91f-43b6-8583-a66dc165782e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:25.191985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a9c1d3-4ad4-4362-9f8b-97b90804a419 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:25.235405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ebdd6-c11e-47f7-8f3b-4385f8341831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:25.236319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ebdd6-c11e-47f7-8f3b-4385f8341831 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:25.236791 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ebdd6-c11e-47f7-8f3b-4385f8341831 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:25.237309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313ebdd6-c11e-47f7-8f3b-4385f8341831 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:25.238187 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ebdd6-c11e-47f7-8f3b-4385f8341831 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:25.239090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ebdd6-c11e-47f7-8f3b-4385f8341831 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:25.240982 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313ebdd6-c11e-47f7-8f3b-4385f8341831 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:25.257899 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 480/950] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:25.263477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ebdd6-c11e-47f7-8f3b-4385f8341831 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:25.265110 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71251ab4-c91f-43b6-8583-a66dc165782e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:25.303645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc1bfe-24d2-4e3d-a64c-e54590ec5abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:25.304173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc1bfe-24d2-4e3d-a64c-e54590ec5abb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:25.304741 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc1bfe-24d2-4e3d-a64c-e54590ec5abb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:25.304850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acc1bfe-24d2-4e3d-a64c-e54590ec5abb None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:25.305445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc1bfe-24d2-4e3d-a64c-e54590ec5abb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:25.306812 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acc1bfe-24d2-4e3d-a64c-e54590ec5abb None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:25.309763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:25.325772 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 471/951] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 19:00:25 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:00:25.370550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:25.398804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313ebdd6-c11e-47f7-8f3b-4385f8341831 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:25.437898 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a0909-073c-4c34-9e89-bbfa9db82d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:25.438936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a0909-073c-4c34-9e89-bbfa9db82d45 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:25.439408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a0909-073c-4c34-9e89-bbfa9db82d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:25.439887 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2a0909-073c-4c34-9e89-bbfa9db82d45 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:25.447930 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a0909-073c-4c34-9e89-bbfa9db82d45 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:25.450053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2a0909-073c-4c34-9e89-bbfa9db82d45 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:25.462991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-cc2a0909-073c-4c34-9e89-bbfa9db82d45 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:25.462991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:25.462991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:25.462991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:25.462991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:25.462991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:25.462991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:25.462991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:25.462991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:25.462991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:25.462991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:25.462991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:25.462991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:00:25.462991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:00:25.462991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:25.463941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:25.465582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:00:25.465582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:00:25.465582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:00:25.465582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:00:25.465582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:00:25.465582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:00:25.465582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:25.465582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:25.468074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 481/952] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:25.474189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc1bfe-24d2-4e3d-a64c-e54590ec5abb None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:25.583754 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acc1bfe-24d2-4e3d-a64c-e54590ec5abb None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:25.622081 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738f95c-a244-493d-aab7-1b96f2bae192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:25.622690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738f95c-a244-493d-aab7-1b96f2bae192 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:25.622971 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738f95c-a244-493d-aab7-1b96f2bae192 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:25.623243 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738f95c-a244-493d-aab7-1b96f2bae192 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:25.623929 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738f95c-a244-493d-aab7-1b96f2bae192 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:25.624487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738f95c-a244-493d-aab7-1b96f2bae192 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:25.625947 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4738f95c-a244-493d-aab7-1b96f2bae192 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:25.642644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 472/953] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:00:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:25.648424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a0909-073c-4c34-9e89-bbfa9db82d45 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:25.756913 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2a0909-073c-4c34-9e89-bbfa9db82d45 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:25.793720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8c659-3c6d-41c7-8632-87c10fb1ed8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:25.794372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8c659-3c6d-41c7-8632-87c10fb1ed8e None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:25.794639 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8c659-3c6d-41c7-8632-87c10fb1ed8e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:25.794899 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8c659-3c6d-41c7-8632-87c10fb1ed8e None admin] PATH_INFO: `/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/dd730375a996433d982b26c8339ec4ee/roles/f37b082e86f54a6a936cf9a83fe956a7` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:25.797107 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8c659-3c6d-41c7-8632-87c10fb1ed8e None admin] RBAC: Authorizing `identity:create_grant(project_id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user_id=dd730375a996433d982b26c8339ec4ee, role_id=f37b082e86f54a6a936cf9a83fe956a7)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:25.798542 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8c659-3c6d-41c7-8632-87c10fb1ed8e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:25.810140 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:25.815288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 482/954] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 19:00:25 2022] PUT /identity/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/dd730375a996433d982b26c8339ec4ee/roles/f37b082e86f54a6a936cf9a83fe956a7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:00:26.595497 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba6ebd-32fe-4885-8866-4c952a227827 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:26.596067 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba6ebd-32fe-4885-8866-4c952a227827 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:26.596067 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ba6ebd-32fe-4885-8866-4c952a227827 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:26.597673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 473/955] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:26.602768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbdea0-99d3-466f-87c9-8947127c50f7 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:26.603169 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbdea0-99d3-466f-87c9-8947127c50f7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:26.603342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cbdea0-99d3-466f-87c9-8947127c50f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:26.623026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-51cbdea0-99d3-466f-87c9-8947127c50f7 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:26.624796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-51cbdea0-99d3-466f-87c9-8947127c50f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:26.693867 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 483/956] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:26.945694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb1f72-ba91-4117-b7de-caeda3fb746d None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:26.945881 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb1f72-ba91-4117-b7de-caeda3fb746d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:26.946239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bb1f72-ba91-4117-b7de-caeda3fb746d None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:26.947954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 474/957] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:26.953205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75e9ab-54b3-4020-b34b-022a9773b0fa None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:26.953520 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75e9ab-54b3-4020-b34b-022a9773b0fa None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:26.953796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b75e9ab-54b3-4020-b34b-022a9773b0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:26.963621 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb1f72-ba91-4117-b7de-caeda3fb746d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:26.972833 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2b75e9ab-54b3-4020-b34b-022a9773b0fa None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:26.974046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2b75e9ab-54b3-4020-b34b-022a9773b0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:27.046364 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:27.053325 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 484/958] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:27.088791 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bb1f72-ba91-4117-b7de-caeda3fb746d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:27.123158 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd4059-b6da-4cdb-9f27-16de52a1d546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:27.123757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd4059-b6da-4cdb-9f27-16de52a1d546 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:27.124034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd4059-b6da-4cdb-9f27-16de52a1d546 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:27.124554 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd4059-b6da-4cdb-9f27-16de52a1d546 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:27.125212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd4059-b6da-4cdb-9f27-16de52a1d546 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:27.126620 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd4059-b6da-4cdb-9f27-16de52a1d546 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:27.129117 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: WARNING keystone.server.flask.common [None req-65cd4059-b6da-4cdb-9f27-16de52a1d546 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 31 19:00:27.143643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.sql.core [None req-65cd4059-b6da-4cdb-9f27-16de52a1d546 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 19:00:27.143643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 19:00:27.143643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [parameters: {'user_id': '634d7f9e62fa4b6fbf95a1fb384c61f6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 19:00:27.143643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81626) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 31 19:00:27.161672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-65cd4059-b6da-4cdb-9f27-16de52a1d546 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 19:00:27.161672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:27.161672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 19:00:27.161672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 19:00:27.161672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 19:00:27.161672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 19:00:27.161672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 19:00:27.161672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 19:00:27.161672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 19:00:27.161672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application conn.query(q) Aug 31 19:00:27.161672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 19:00:27.161672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 19:00:27.161672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 19:00:27.161672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result.read() Aug 31 19:00:27.161672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 19:00:27.163502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application next(self.gen) Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application yield resource Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application next(self.gen) Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.session.rollback() Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.force_reraise() Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise self.value Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application session.commit() Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 31 19:00:27.167301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._prepare_impl() Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.session.flush() Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._flush(objects) Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application compat.raise_( Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application flush_context.execute() Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rec.execute(self) Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 31 19:00:27.171705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 31 19:00:27.173921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 31 19:00:27.173921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 31 19:00:27.173921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 31 19:00:27.173921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 31 19:00:27.173921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 31 19:00:27.173921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 31 19:00:27.173921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 31 19:00:27.173921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 31 19:00:27.173921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 31 19:00:27.173921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 31 19:00:27.173921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 31 19:00:27.173921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 31 19:00:27.173921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 31 19:00:27.173921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 31 19:00:27.173921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 31 19:00:27.173921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception Aug 31 19:00:27.174810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 31 19:00:27.174810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 31 19:00:27.174810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 31 19:00:27.174810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 31 19:00:27.174810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 31 19:00:27.174810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result = self._query(query) Aug 31 19:00:27.174810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 31 19:00:27.174810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application conn.query(q) Aug 31 19:00:27.174810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 31 19:00:27.174810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 31 19:00:27.174810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 31 19:00:27.174810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application result.read() Aug 31 19:00:27.174810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 31 19:00:27.174810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 31 19:00:27.174810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 31 19:00:27.174810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 31 19:00:27.175677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 31 19:00:27.175677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 31 19:00:27.175677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 31 19:00:27.175677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 31 19:00:27.175677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 31 19:00:27.175677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 31 19:00:27.175677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [parameters: {'user_id': '634d7f9e62fa4b6fbf95a1fb384c61f6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 31 19:00:27.175677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 31 19:00:27.175677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:27.175677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:27.175677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:27.175677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:27.175677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:27.175677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:27.175677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 31 19:00:27.176509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 31 19:00:27.182123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 31 19:00:27.182123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 31 19:00:27.182123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 31 19:00:27.182123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 31 19:00:27.182123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 31 19:00:27.182123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:27.182123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 475/959] 10.222.0.65 () {60 vars in 1131 bytes} [Wed Aug 31 19:00:26 2022] POST /identity/v3/users => generated 160 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:27.183486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75e9ab-54b3-4020-b34b-022a9773b0fa None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:27.237772 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:27.283813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b75e9ab-54b3-4020-b34b-022a9773b0fa None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:27.319949 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8ffd9d-49cb-4f22-9b87-2bfefc12519e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:27.320580 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ffd9d-49cb-4f22-9b87-2bfefc12519e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:27.320859 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ffd9d-49cb-4f22-9b87-2bfefc12519e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:27.321128 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8ffd9d-49cb-4f22-9b87-2bfefc12519e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:27.329173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ffd9d-49cb-4f22-9b87-2bfefc12519e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:27.330649 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8ffd9d-49cb-4f22-9b87-2bfefc12519e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:27.336909 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1c9a7-592a-485d-bd6d-5fe8e1f25416 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:27.337213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1c9a7-592a-485d-bd6d-5fe8e1f25416 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:27.337494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1c9a7-592a-485d-bd6d-5fe8e1f25416 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:27.343605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-9b8ffd9d-49cb-4f22-9b87-2bfefc12519e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:00:27.343605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:27.343605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:27.343605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:27.343605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:27.343605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:27.343605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:27.343605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:27.343605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:27.343605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:27.343605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:27.343605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:27.343605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:00:27.343605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:00:27.343605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:00:27.343605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:27.344554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:27.345623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:27.345623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:27.345623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:00:27.345623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:00:27.345623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:00:27.345623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:00:27.345623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:00:27.345623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:00:27.345623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 31 19:00:27.345623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:27.345623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 485/961] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:00:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:27.346249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 476/961] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:27.347348 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dda4ad-727c-4a47-b117-c6d0ca40ec9e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:27.347685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dda4ad-727c-4a47-b117-c6d0ca40ec9e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:27.347962 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dda4ad-727c-4a47-b117-c6d0ca40ec9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:27.352220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1c9a7-592a-485d-bd6d-5fe8e1f25416 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:27.368186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-d2dda4ad-727c-4a47-b117-c6d0ca40ec9e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:27.369427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d2dda4ad-727c-4a47-b117-c6d0ca40ec9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:27.446875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 486/962] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:27 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:00:27.447783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dda4ad-727c-4a47-b117-c6d0ca40ec9e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:27.470202 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1c9a7-592a-485d-bd6d-5fe8e1f25416 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:27.506617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:27.510468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:27.531072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df68b0-8571-4d08-a0c5-8f00dfb4384d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:27.532155 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df68b0-8571-4d08-a0c5-8f00dfb4384d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:27.532714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df68b0-8571-4d08-a0c5-8f00dfb4384d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:27.533331 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df68b0-8571-4d08-a0c5-8f00dfb4384d None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:27.534911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df68b0-8571-4d08-a0c5-8f00dfb4384d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:27.535582 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df68b0-8571-4d08-a0c5-8f00dfb4384d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:27.538916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19df68b0-8571-4d08-a0c5-8f00dfb4384d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:27.564277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dda4ad-727c-4a47-b117-c6d0ca40ec9e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:27.602143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9c2a74-47f0-4dde-9961-442696ee5508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:27.602842 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c2a74-47f0-4dde-9961-442696ee5508 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:27.603387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c2a74-47f0-4dde-9961-442696ee5508 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:27.603387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9c2a74-47f0-4dde-9961-442696ee5508 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:27.604171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c2a74-47f0-4dde-9961-442696ee5508 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:27.605726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9c2a74-47f0-4dde-9961-442696ee5508 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:27.607673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 477/963] 10.222.0.65 () {56 vars in 1101 bytes} [Wed Aug 31 19:00:27 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:27.634967 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 487/964] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 19:00:27 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:00:27.768699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df68b0-8571-4d08-a0c5-8f00dfb4384d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:27.862267 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cd4f1-7be5-4304-ac42-0c18a988e769 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:27.862548 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cd4f1-7be5-4304-ac42-0c18a988e769 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:27.862817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6cd4f1-7be5-4304-ac42-0c18a988e769 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:27.864417 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 488/965] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:00:27.869805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce2e04-e6ff-4e61-aed5-84c805807999 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:27.870222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce2e04-e6ff-4e61-aed5-84c805807999 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:27.870499 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce2e04-e6ff-4e61-aed5-84c805807999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:27.883743 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df68b0-8571-4d08-a0c5-8f00dfb4384d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:27.894565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f7ce2e04-e6ff-4e61-aed5-84c805807999 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:27.896715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f7ce2e04-e6ff-4e61-aed5-84c805807999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:27.935278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3f464-b513-42f1-9516-bf1ebc9a0b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:27.935914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3f464-b513-42f1-9516-bf1ebc9a0b0c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:27.936197 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3f464-b513-42f1-9516-bf1ebc9a0b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:27.936467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf3f464-b513-42f1-9516-bf1ebc9a0b0c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:27.937044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3f464-b513-42f1-9516-bf1ebc9a0b0c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:27.938424 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf3f464-b513-42f1-9516-bf1ebc9a0b0c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-bbf3f464-b513-42f1-9516-bf1ebc9a0b0c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:27.947376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:27.948393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:27.948393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:27.948393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:27.948393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:27.948393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:27.948393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:27.948393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:27.948393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:27.948393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:27.948393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:27.948393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:27.948393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:27.948393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:27.948393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:27.948393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:27.948393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:27.949830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:27.949830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:27.949830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:27.949830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:27.949830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:27.949830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:27.949830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:27.949830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:27.949830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:27.949830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:27.949830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:27.949830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:27.949830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:00:27.949830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:27.949830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 478/966] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 19:00:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:27.955449 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3f464-b513-42f1-9516-bf1ebc9a0b0c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:27.972792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 489/967] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:00:28.003061 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:28.064282 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf3f464-b513-42f1-9516-bf1ebc9a0b0c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:28.099951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a55e7-5b8a-4e8a-9a15-e992be00a457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:28.100552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a55e7-5b8a-4e8a-9a15-e992be00a457 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:28.100830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a55e7-5b8a-4e8a-9a15-e992be00a457 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:28.101123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5a55e7-5b8a-4e8a-9a15-e992be00a457 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:28.101772 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a55e7-5b8a-4e8a-9a15-e992be00a457 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:28.102073 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a55e7-5b8a-4e8a-9a15-e992be00a457 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:28.103428 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5a55e7-5b8a-4e8a-9a15-e992be00a457 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:28.109055 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 479/968] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 19:00:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:28.114683 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce2e04-e6ff-4e61-aed5-84c805807999 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:28.162118 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:28.228777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce2e04-e6ff-4e61-aed5-84c805807999 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:28.242149 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a55e7-5b8a-4e8a-9a15-e992be00a457 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:28.264744 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccec356-e2bf-4184-b05d-84a977368a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:28.265379 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccec356-e2bf-4184-b05d-84a977368a21 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:28.265651 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccec356-e2bf-4184-b05d-84a977368a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:28.265917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccec356-e2bf-4184-b05d-84a977368a21 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:28.266535 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccec356-e2bf-4184-b05d-84a977368a21 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:28.266827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccec356-e2bf-4184-b05d-84a977368a21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:28.268168 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccec356-e2bf-4184-b05d-84a977368a21 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:28.278271 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 490/969] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 19:00:28 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:28.280344 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccec356-e2bf-4184-b05d-84a977368a21 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:28.363658 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5a55e7-5b8a-4e8a-9a15-e992be00a457 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:28.366960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:28.387334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccec356-e2bf-4184-b05d-84a977368a21 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:28.390207 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:28.401301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a1d1c-5bad-47ca-b673-a975aebb3b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:28.401926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a1d1c-5bad-47ca-b673-a975aebb3b14 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:28.402231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a1d1c-5bad-47ca-b673-a975aebb3b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:28.402502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a1d1c-5bad-47ca-b673-a975aebb3b14 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:28.412482 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a1d1c-5bad-47ca-b673-a975aebb3b14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:00:28.417683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-215a1d1c-5bad-47ca-b673-a975aebb3b14 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:00:28.417683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:28.417683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:28.417683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:28.417683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:28.417683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:28.417683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:28.417683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:28.417683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:28.417683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:28.417683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:28.417683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:28.417683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:00:28.417683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:00:28.417683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:00:28.417683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:00:28.419131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:00:28.419131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:00:28.419131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:00:28.419131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:00:28.419131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:28.419131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:28.419131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:28.419131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:28.419131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:28.419131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:28.419131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:28.419131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:28.419131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:28.419131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:28.419131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:28.419131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:28.419985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:28.419985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:28.419985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:28.419985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:28.419985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:00:28.419985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:00:28.419985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:00:28.419985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:00:28.419985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:00:28.419985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:00:28.419985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:00:28.419985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:28.419985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 480/970] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 31 19:00:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:28.425169 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a1d1c-5bad-47ca-b673-a975aebb3b14 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:28.429519 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473ff82-acae-4102-a632-bdaa61831b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:28.430312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473ff82-acae-4102-a632-bdaa61831b65 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:28.430732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473ff82-acae-4102-a632-bdaa61831b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:28.431156 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8473ff82-acae-4102-a632-bdaa61831b65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:28.432044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473ff82-acae-4102-a632-bdaa61831b65 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:28.434276 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8473ff82-acae-4102-a632-bdaa61831b65 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:28.456750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 491/971] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:00:28 2022] POST /identity/v3/endpoints => generated 349 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:00:28.542208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a1d1c-5bad-47ca-b673-a975aebb3b14 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:28.585295 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d285d8-2972-4e82-832a-d33b417d2d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:28.586216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d285d8-2972-4e82-832a-d33b417d2d3f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:28.586787 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d285d8-2972-4e82-832a-d33b417d2d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:28.587258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d285d8-2972-4e82-832a-d33b417d2d3f None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:28.588096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d285d8-2972-4e82-832a-d33b417d2d3f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:28.588575 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d285d8-2972-4e82-832a-d33b417d2d3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:28.590686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d285d8-2972-4e82-832a-d33b417d2d3f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:28.606561 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 481/972] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 31 19:00:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:28.608147 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473ff82-acae-4102-a632-bdaa61831b65 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:28.707207 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8473ff82-acae-4102-a632-bdaa61831b65 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:28.740224 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477595f4-4926-4945-8efa-aabf80f11518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:28.740844 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477595f4-4926-4945-8efa-aabf80f11518 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:28.741112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477595f4-4926-4945-8efa-aabf80f11518 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:28.741398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477595f4-4926-4945-8efa-aabf80f11518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:28.751950 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477595f4-4926-4945-8efa-aabf80f11518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:28.753397 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477595f4-4926-4945-8efa-aabf80f11518 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:28.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-477595f4-4926-4945-8efa-aabf80f11518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:28.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:28.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:28.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:28.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:28.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:28.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:28.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:28.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:28.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:28.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:28.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:28.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:28.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:28.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:28.767705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:28.768785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:28.774319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:28.774319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:28.774319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:28.774319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 492/973] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:28.777655 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d285d8-2972-4e82-832a-d33b417d2d3f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:28.802407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:28.878503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d285d8-2972-4e82-832a-d33b417d2d3f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:28.912455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfec9ee-eb63-44c6-9d86-33258525533b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:28.913049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfec9ee-eb63-44c6-9d86-33258525533b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:28.913364 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfec9ee-eb63-44c6-9d86-33258525533b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:28.913634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfec9ee-eb63-44c6-9d86-33258525533b None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:28.914258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfec9ee-eb63-44c6-9d86-33258525533b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:28.914551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfec9ee-eb63-44c6-9d86-33258525533b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:28.915883 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfec9ee-eb63-44c6-9d86-33258525533b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:28.927634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 482/974] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:28.932925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477595f4-4926-4945-8efa-aabf80f11518 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:28.948165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:29.035552 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477595f4-4926-4945-8efa-aabf80f11518 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:29.067828 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd09b6-3e3c-4d4a-8db0-cc7eeecce36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:29.068448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd09b6-3e3c-4d4a-8db0-cc7eeecce36d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:29.068721 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd09b6-3e3c-4d4a-8db0-cc7eeecce36d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:29.068984 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd09b6-3e3c-4d4a-8db0-cc7eeecce36d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:29.076959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd09b6-3e3c-4d4a-8db0-cc7eeecce36d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:29.078449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd09b6-3e3c-4d4a-8db0-cc7eeecce36d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:29.089353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-12bd09b6-3e3c-4d4a-8db0-cc7eeecce36d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:00:29.089353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:29.089353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:29.089353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:29.089353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:29.089353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:29.089353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:29.089353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:29.089353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:29.089353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:29.089353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:29.089353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:29.089353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:00:29.089353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:00:29.089353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:00:29.089353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:29.090916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:29.091858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:29.091858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:29.091858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:00:29.091858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:00:29.091858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:00:29.091858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:00:29.091858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:00:29.091858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:00:29.091858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 31 19:00:29.091858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:29.097179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 493/975] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 19:00:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:29.097352 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfec9ee-eb63-44c6-9d86-33258525533b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:29.181978 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:29.206318 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfec9ee-eb63-44c6-9d86-33258525533b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:29.241180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99230b79-3fdc-4f7a-8abd-2aba4c4a7cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:29.241811 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99230b79-3fdc-4f7a-8abd-2aba4c4a7cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:29.242094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99230b79-3fdc-4f7a-8abd-2aba4c4a7cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:29.242364 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99230b79-3fdc-4f7a-8abd-2aba4c4a7cb2 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:29.243084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99230b79-3fdc-4f7a-8abd-2aba4c4a7cb2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:29.243387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99230b79-3fdc-4f7a-8abd-2aba4c4a7cb2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:29.244796 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99230b79-3fdc-4f7a-8abd-2aba4c4a7cb2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:29.267925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 483/976] 10.222.0.65 () {56 vars in 1149 bytes} [Wed Aug 31 19:00:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:29.269212 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd09b6-3e3c-4d4a-8db0-cc7eeecce36d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:29.340291 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:29.371871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd09b6-3e3c-4d4a-8db0-cc7eeecce36d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:29.404911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42db71fc-4ded-4fbf-af4d-11d2d89d466e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:29.405533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db71fc-4ded-4fbf-af4d-11d2d89d466e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:29.405803 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db71fc-4ded-4fbf-af4d-11d2d89d466e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:29.406066 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42db71fc-4ded-4fbf-af4d-11d2d89d466e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:29.474772 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db71fc-4ded-4fbf-af4d-11d2d89d466e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:29.476212 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42db71fc-4ded-4fbf-af4d-11d2d89d466e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:29.547904 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:29.586547 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 494/977] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:00:29 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:00:29.587725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99230b79-3fdc-4f7a-8abd-2aba4c4a7cb2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:29.603101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df06419-0ed0-4606-90ad-6633f2b2ba58 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:29.603354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df06419-0ed0-4606-90ad-6633f2b2ba58 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:29.603977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df06419-0ed0-4606-90ad-6633f2b2ba58 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:29.606113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 495/978] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:00:29.610758 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93588f-1364-48f0-9c6f-e8267d77e73e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:29.611323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93588f-1364-48f0-9c6f-e8267d77e73e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:29.611323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a93588f-1364-48f0-9c6f-e8267d77e73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:29.629983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4a93588f-1364-48f0-9c6f-e8267d77e73e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:29.631162 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4a93588f-1364-48f0-9c6f-e8267d77e73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:29.694318 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99230b79-3fdc-4f7a-8abd-2aba4c4a7cb2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:29.701222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 496/979] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:00:29.732841 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30c34c-ac2d-4193-b56a-329d1d806da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:29.733695 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30c34c-ac2d-4193-b56a-329d1d806da0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:29.733983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30c34c-ac2d-4193-b56a-329d1d806da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:29.734261 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30c34c-ac2d-4193-b56a-329d1d806da0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:29.743011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:29.745388 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30c34c-ac2d-4193-b56a-329d1d806da0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:29.747488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30c34c-ac2d-4193-b56a-329d1d806da0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:29.764032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-6e30c34c-ac2d-4193-b56a-329d1d806da0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:29.764032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:29.764032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:29.764032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:29.764032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:29.764032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:29.764032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:29.764032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:29.764032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:29.764032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:29.764032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:29.764032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:29.764032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:29.764032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:29.764032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:29.764032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:29.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:29.766631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:29.766631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:29.766631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:29.766631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 484/980] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:29.770669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93588f-1364-48f0-9c6f-e8267d77e73e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:29.877724 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a93588f-1364-48f0-9c6f-e8267d77e73e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:29.879171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:29.912754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2f56e-5d76-4457-9e83-efa7baa056e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:29.913394 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2f56e-5d76-4457-9e83-efa7baa056e0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:29.913667 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2f56e-5d76-4457-9e83-efa7baa056e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:29.913928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e2f56e-5d76-4457-9e83-efa7baa056e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:29.914552 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2f56e-5d76-4457-9e83-efa7baa056e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:29.914842 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2f56e-5d76-4457-9e83-efa7baa056e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:29.916184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e2f56e-5d76-4457-9e83-efa7baa056e0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:29.928173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 497/981] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:29.934271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30c34c-ac2d-4193-b56a-329d1d806da0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:29.971163 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2f56e-5d76-4457-9e83-efa7baa056e0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:30.048212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30c34c-ac2d-4193-b56a-329d1d806da0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:30.081638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e2f56e-5d76-4457-9e83-efa7baa056e0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:30.084165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902f6ea-f933-40f8-b760-1592aafb668a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:30.085626 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902f6ea-f933-40f8-b760-1592aafb668a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:30.085912 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902f6ea-f933-40f8-b760-1592aafb668a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:30.086185 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6902f6ea-f933-40f8-b760-1592aafb668a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:30.093686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902f6ea-f933-40f8-b760-1592aafb668a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:30.095103 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6902f6ea-f933-40f8-b760-1592aafb668a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:30.108488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-6902f6ea-f933-40f8-b760-1592aafb668a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:30.108488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:30.108488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:30.108488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:30.108488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:30.108488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:30.108488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:30.108488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:30.108488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:30.108488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:30.108488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:30.108488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:30.108488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:00:30.108488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:00:30.108488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:30.109553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:30.110461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:00:30.110461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:00:30.110461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:00:30.110461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:00:30.110461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:00:30.110461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:00:30.110461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:30.110461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:30.110461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 485/982] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:30.115411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902f6ea-f933-40f8-b760-1592aafb668a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:30.119567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca5e67-b791-4605-9ae8-7958dea76f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:30.120190 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca5e67-b791-4605-9ae8-7958dea76f65 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:30.120461 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca5e67-b791-4605-9ae8-7958dea76f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:30.120726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ca5e67-b791-4605-9ae8-7958dea76f65 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:30.121343 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca5e67-b791-4605-9ae8-7958dea76f65 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:30.122683 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ca5e67-b791-4605-9ae8-7958dea76f65 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-01ca5e67-b791-4605-9ae8-7958dea76f65 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:30.131699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:30.132691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:30.132691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:30.132691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:30.132691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:30.132691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:30.132691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:30.132691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:30.132691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:30.132691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:30.132691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:30.132691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:30.132691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:30.132691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:30.132691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:30.132691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:30.132691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:30.133579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:30.133579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:30.133579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:30.133579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:30.133579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:30.133579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:30.133579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:30.133579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:30.133579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:30.133579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:30.133579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:30.133579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:30.133579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:00:30.133579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:30.133579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 498/983] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:30.138497 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca5e67-b791-4605-9ae8-7958dea76f65 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:30.167606 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:30.193605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:30.247146 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6902f6ea-f933-40f8-b760-1592aafb668a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:30.260665 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ca5e67-b791-4605-9ae8-7958dea76f65 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:30.283814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd904ec-17f8-4b8f-a78d-fd3a5a0b45b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:30.284497 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd904ec-17f8-4b8f-a78d-fd3a5a0b45b8 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:30.284781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd904ec-17f8-4b8f-a78d-fd3a5a0b45b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:30.285051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd904ec-17f8-4b8f-a78d-fd3a5a0b45b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:30.285773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd904ec-17f8-4b8f-a78d-fd3a5a0b45b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:30.286064 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd904ec-17f8-4b8f-a78d-fd3a5a0b45b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:30.287473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd904ec-17f8-4b8f-a78d-fd3a5a0b45b8 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:30.297696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a824f-7f65-4ef9-8248-fe119299c00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:30.298423 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a824f-7f65-4ef9-8248-fe119299c00f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:30.298595 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a824f-7f65-4ef9-8248-fe119299c00f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:30.298864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a824f-7f65-4ef9-8248-fe119299c00f None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:30.299493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a824f-7f65-4ef9-8248-fe119299c00f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:30.299791 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a824f-7f65-4ef9-8248-fe119299c00f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:30.306994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a824f-7f65-4ef9-8248-fe119299c00f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:30.307133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 486/984] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:00:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:30.307133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd904ec-17f8-4b8f-a78d-fd3a5a0b45b8 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:30.310761 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 499/985] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 19:00:30 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:30.311513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a824f-7f65-4ef9-8248-fe119299c00f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:30.404296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:30.414187 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a824f-7f65-4ef9-8248-fe119299c00f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:30.414632 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd904ec-17f8-4b8f-a78d-fd3a5a0b45b8 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:30.427714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:30.447572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed6b72-7a0e-4fd6-a4b5-a3dce19a4b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:30.448178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed6b72-7a0e-4fd6-a4b5-a3dce19a4b86 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:30.448454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed6b72-7a0e-4fd6-a4b5-a3dce19a4b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:30.448721 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed6b72-7a0e-4fd6-a4b5-a3dce19a4b86 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:30.449887 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed6b72-7a0e-4fd6-a4b5-a3dce19a4b86 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:30.450183 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed6b72-7a0e-4fd6-a4b5-a3dce19a4b86 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:30.451086 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff5c253-a52c-464f-a1e1-d97619387b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:30.451535 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed6b72-7a0e-4fd6-a4b5-a3dce19a4b86 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:30.451684 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5c253-a52c-464f-a1e1-d97619387b7f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:30.451965 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5c253-a52c-464f-a1e1-d97619387b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:30.452252 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff5c253-a52c-464f-a1e1-d97619387b7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:30.452948 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5c253-a52c-464f-a1e1-d97619387b7f None admin] RBAC: Adding query filter params (role.id=f37b082e86f54a6a936cf9a83fe956a7, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=dd730375a996433d982b26c8339ec4ee) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:30.453281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5c253-a52c-464f-a1e1-d97619387b7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:30.454676 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff5c253-a52c-464f-a1e1-d97619387b7f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:30.456308 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 500/986] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 19:00:30 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:30.457552 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6d86e-ae1d-480d-9f49-67b479d2a16f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:30.457835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6d86e-ae1d-480d-9f49-67b479d2a16f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:30.458098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6d86e-ae1d-480d-9f49-67b479d2a16f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:30.459293 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 501/987] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:30 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:30.461442 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 487/988] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:00:30 2022] GET /identity/v3/role_assignments?user.id=dd730375a996433d982b26c8339ec4ee&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=f37b082e86f54a6a936cf9a83fe956a7 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:30.464034 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6d86e-ae1d-480d-9f49-67b479d2a16f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:30.464539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649db54-45d0-401b-a5f8-5ed92a50a4d4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:30.465427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649db54-45d0-401b-a5f8-5ed92a50a4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:30.465705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649db54-45d0-401b-a5f8-5ed92a50a4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:30.484939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0649db54-45d0-401b-a5f8-5ed92a50a4d4 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:30.486584 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0649db54-45d0-401b-a5f8-5ed92a50a4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:30.562159 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 488/989] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:00:30.588741 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6d86e-ae1d-480d-9f49-67b479d2a16f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:30.627644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f684b-4e77-4845-87df-c6ea641fdc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:30.628272 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f684b-4e77-4845-87df-c6ea641fdc1f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:30.628512 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f684b-4e77-4845-87df-c6ea641fdc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:30.628784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f684b-4e77-4845-87df-c6ea641fdc1f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:30.629515 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f684b-4e77-4845-87df-c6ea641fdc1f None admin] RBAC: Adding query filter params (interface=public, service_id=2327d00ed6a342c8918afc614f9c8c18, region_id=RegionOne) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:30.629814 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f684b-4e77-4845-87df-c6ea641fdc1f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:30.631171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f684b-4e77-4845-87df-c6ea641fdc1f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:30.637812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 502/990] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 31 19:00:30 2022] GET /identity/v3/endpoints?service_id=2327d00ed6a342c8918afc614f9c8c18&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:30.643228 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0649db54-45d0-401b-a5f8-5ed92a50a4d4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:30.746729 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0649db54-45d0-401b-a5f8-5ed92a50a4d4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:30.755533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:30.783761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba38f7-cebf-4930-aa82-6e4cf81483ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:30.784360 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba38f7-cebf-4930-aa82-6e4cf81483ca None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:30.784633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba38f7-cebf-4930-aa82-6e4cf81483ca None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:30.784901 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ba38f7-cebf-4930-aa82-6e4cf81483ca None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:30.785600 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba38f7-cebf-4930-aa82-6e4cf81483ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:30.786955 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ba38f7-cebf-4930-aa82-6e4cf81483ca None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:30.801277 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 489/991] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 19:00:30 2022] GET /identity/v3/services => generated 1727 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:00:30.828866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f684b-4e77-4845-87df-c6ea641fdc1f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:30.850467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:30.941546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f684b-4e77-4845-87df-c6ea641fdc1f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:30.977996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e841b-f8cf-496a-952f-05c1dac1def8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:30.978626 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e841b-f8cf-496a-952f-05c1dac1def8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:30.978894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e841b-f8cf-496a-952f-05c1dac1def8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:30.979160 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454e841b-f8cf-496a-952f-05c1dac1def8 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:30.979731 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e841b-f8cf-496a-952f-05c1dac1def8 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:30.981111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454e841b-f8cf-496a-952f-05c1dac1def8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-454e841b-f8cf-496a-952f-05c1dac1def8 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:30.989810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:30.990778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:30.990778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:30.990778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:30.990778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:30.990778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:30.990778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:30.990778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:30.990778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:30.990778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:30.990778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:30.990778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:30.990778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:30.990778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:30.990778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:30.990778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:30.990778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:30.991617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:30.991617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:30.991617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:30.991617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:30.991617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:30.991617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:30.991617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:30.991617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:30.991617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:30.991617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:30.991617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:30.991617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:30.991617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:00:30.991617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:30.991617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 503/992] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 19:00:30 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:30.997479 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba38f7-cebf-4930-aa82-6e4cf81483ca None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:31.099857 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ba38f7-cebf-4930-aa82-6e4cf81483ca None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:31.134971 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327d7a6-6422-4461-8d18-69ecfbca1275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:31.135571 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327d7a6-6422-4461-8d18-69ecfbca1275 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:31.135847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327d7a6-6422-4461-8d18-69ecfbca1275 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:31.136115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e327d7a6-6422-4461-8d18-69ecfbca1275 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:31.136767 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327d7a6-6422-4461-8d18-69ecfbca1275 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:31.137063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327d7a6-6422-4461-8d18-69ecfbca1275 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:31.138579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e327d7a6-6422-4461-8d18-69ecfbca1275 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:31.148208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 490/993] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 19:00:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:31.149466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e841b-f8cf-496a-952f-05c1dac1def8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:31.241258 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:31.260827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454e841b-f8cf-496a-952f-05c1dac1def8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:31.295195 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5f2b8-13ea-4500-9011-11851d8e1e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:31.295792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5f2b8-13ea-4500-9011-11851d8e1e91 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:31.296063 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5f2b8-13ea-4500-9011-11851d8e1e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:31.296327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f5f2b8-13ea-4500-9011-11851d8e1e91 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:31.296942 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5f2b8-13ea-4500-9011-11851d8e1e91 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:31.297262 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5f2b8-13ea-4500-9011-11851d8e1e91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:31.298605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f5f2b8-13ea-4500-9011-11851d8e1e91 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:31.303685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 504/994] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 19:00:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:31.309864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327d7a6-6422-4461-8d18-69ecfbca1275 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:31.344015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:31.411107 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e327d7a6-6422-4461-8d18-69ecfbca1275 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:31.444797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab419c8-9eea-468a-809d-858d9f33e847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:31.445419 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab419c8-9eea-468a-809d-858d9f33e847 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:31.445695 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab419c8-9eea-468a-809d-858d9f33e847 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:31.445961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab419c8-9eea-468a-809d-858d9f33e847 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:31.446656 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab419c8-9eea-468a-809d-858d9f33e847 None admin] RBAC: Adding query filter params (interface=admin, service_id=8b8f7950ecc3420e900acee60578266c, region_id=RegionOne) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:31.446957 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab419c8-9eea-468a-809d-858d9f33e847 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:31.448306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab419c8-9eea-468a-809d-858d9f33e847 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:31.453555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 491/995] 10.222.0.65 () {56 vars in 1249 bytes} [Wed Aug 31 19:00:31 2022] GET /identity/v3/endpoints?service_id=8b8f7950ecc3420e900acee60578266c&interface=admin®ion_id=RegionOne => generated 193 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:31.458901 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5f2b8-13ea-4500-9011-11851d8e1e91 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:31.560803 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f5f2b8-13ea-4500-9011-11851d8e1e91 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:31.596938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf490a1-5b14-457d-a25e-f7c9942815fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:31.597695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf490a1-5b14-457d-a25e-f7c9942815fe None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:31.597970 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf490a1-5b14-457d-a25e-f7c9942815fe None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:31.598242 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf490a1-5b14-457d-a25e-f7c9942815fe None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:31.598839 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf490a1-5b14-457d-a25e-f7c9942815fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:31.600201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf490a1-5b14-457d-a25e-f7c9942815fe None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:31.613211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 505/996] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 19:00:31 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:00:32.408500 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689f4bb-90e6-42e3-b320-73c20e3ef67d None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:32.409058 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689f4bb-90e6-42e3-b320-73c20e3ef67d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:32.409058 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f689f4bb-90e6-42e3-b320-73c20e3ef67d None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:32.410445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 492/997] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:32.415618 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27938f86-22d0-4839-9019-76c6b0ce9e92 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:32.416041 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27938f86-22d0-4839-9019-76c6b0ce9e92 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:32.416182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27938f86-22d0-4839-9019-76c6b0ce9e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:32.434667 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-27938f86-22d0-4839-9019-76c6b0ce9e92 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:32.435838 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-27938f86-22d0-4839-9019-76c6b0ce9e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:32.439415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:32.509213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 506/998] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:32.729115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71965b24-774a-45f2-bfa2-06f00309a416 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:32.729510 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71965b24-774a-45f2-bfa2-06f00309a416 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:32.729703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71965b24-774a-45f2-bfa2-06f00309a416 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:32.731713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 493/999] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:32.736587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b51c7-b5b0-48b8-b1c6-f69ab2ad75d8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:32.736873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b51c7-b5b0-48b8-b1c6-f69ab2ad75d8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:32.737142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080b51c7-b5b0-48b8-b1c6-f69ab2ad75d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:32.756150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-080b51c7-b5b0-48b8-b1c6-f69ab2ad75d8 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:32.757353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-080b51c7-b5b0-48b8-b1c6-f69ab2ad75d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:32.778045 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71965b24-774a-45f2-bfa2-06f00309a416 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:32.858658 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 507/1000] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:32 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:32.903517 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:32.909686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71965b24-774a-45f2-bfa2-06f00309a416 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:32.945793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7962c15-c83c-4c6f-9d3d-d7c1640a0335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:32.946628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7962c15-c83c-4c6f-9d3d-d7c1640a0335 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:32.947046 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7962c15-c83c-4c6f-9d3d-d7c1640a0335 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:32.947504 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7962c15-c83c-4c6f-9d3d-d7c1640a0335 None admin] PATH_INFO: `/v3/services/image` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:32.948236 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7962c15-c83c-4c6f-9d3d-d7c1640a0335 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:32.949835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7962c15-c83c-4c6f-9d3d-d7c1640a0335 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-d7962c15-c83c-4c6f-9d3d-d7c1640a0335 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:32.959257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:32.960404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:32.960404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:32.960404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:32.960404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:32.960404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:32.960404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:32.960404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:32.960404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:32.960404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:32.960404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:32.960404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:32.960404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:32.960404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:32.960404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:32.960404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:32.960404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:32.961412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:32.961412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:32.961412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:32.961412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:32.961412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:32.961412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:32.961412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:32.961412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:32.961412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:32.961412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:32.961412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:32.961412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:32.961412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:00:32.961412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:32.963722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 494/1001] 10.222.0.65 () {56 vars in 1105 bytes} [Wed Aug 31 19:00:32 2022] GET /identity/v3/services/image => generated 86 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:32.970134 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b51c7-b5b0-48b8-b1c6-f69ab2ad75d8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:33.041165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:33.074275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080b51c7-b5b0-48b8-b1c6-f69ab2ad75d8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:33.108490 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ef822-7b89-440b-b895-7afb9e488d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:33.109093 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ef822-7b89-440b-b895-7afb9e488d68 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:33.109392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ef822-7b89-440b-b895-7afb9e488d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:33.109659 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ef822-7b89-440b-b895-7afb9e488d68 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:33.110277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ef822-7b89-440b-b895-7afb9e488d68 None admin] RBAC: Adding query filter params (name=image) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:33.110571 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ef822-7b89-440b-b895-7afb9e488d68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:33.111913 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ef822-7b89-440b-b895-7afb9e488d68 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:33.122127 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 508/1002] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:33.123684 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7962c15-c83c-4c6f-9d3d-d7c1640a0335 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:33.151786 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ef822-7b89-440b-b895-7afb9e488d68 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:33.243644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7962c15-c83c-4c6f-9d3d-d7c1640a0335 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:33.279877 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ef822-7b89-440b-b895-7afb9e488d68 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:33.282635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb69d9-0e84-4d14-91cb-3935cda98b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:33.283317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb69d9-0e84-4d14-91cb-3935cda98b62 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:33.283667 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb69d9-0e84-4d14-91cb-3935cda98b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:33.284010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb69d9-0e84-4d14-91cb-3935cda98b62 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:33.284741 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb69d9-0e84-4d14-91cb-3935cda98b62 None admin] RBAC: Adding query filter params (type=image) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:33.285111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb69d9-0e84-4d14-91cb-3935cda98b62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:33.286501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb69d9-0e84-4d14-91cb-3935cda98b62 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:33.301322 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 495/1003] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:33 2022] GET /identity/v3/services?type=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:33.317784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5589eea-a613-4161-bca4-e49738da4d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:33.318448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5589eea-a613-4161-bca4-e49738da4d5d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:33.318715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5589eea-a613-4161-bca4-e49738da4d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:33.318979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5589eea-a613-4161-bca4-e49738da4d5d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:33.319565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5589eea-a613-4161-bca4-e49738da4d5d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:33.320943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5589eea-a613-4161-bca4-e49738da4d5d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:33.323274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-b5589eea-a613-4161-bca4-e49738da4d5d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:33.332872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:33.337342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:33.337342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:33.337342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:33.337342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:33.337342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:33.337342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:33.337342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:33.337342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:33.337342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:33.337342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:33.337342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:33.337342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:33.337342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:33.337342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:33.337342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:33.337342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:33.338283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:33.338283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:33.338283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:33.338283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:33.338283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:33.338283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:33.338283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:33.338283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:33.338283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:33.338283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:33.338283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:33.338283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:33.338283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 31 19:00:33.338283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:33.338283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 509/1004] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:33.340952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb69d9-0e84-4d14-91cb-3935cda98b62 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:33.371955 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:33.454240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb69d9-0e84-4d14-91cb-3935cda98b62 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:33.488399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc3f35b-5ed3-4787-b6ef-956eff6c0671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:33.488997 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3f35b-5ed3-4787-b6ef-956eff6c0671 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:33.489292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3f35b-5ed3-4787-b6ef-956eff6c0671 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:33.489560 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc3f35b-5ed3-4787-b6ef-956eff6c0671 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:33.490173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3f35b-5ed3-4787-b6ef-956eff6c0671 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:33.490466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3f35b-5ed3-4787-b6ef-956eff6c0671 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:33.491808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc3f35b-5ed3-4787-b6ef-956eff6c0671 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:33.496770 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 496/1005] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 19:00:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:33.502968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5589eea-a613-4161-bca4-e49738da4d5d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:33.509231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae6012-69da-4251-b85a-768e05a8b871 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:33.509688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae6012-69da-4251-b85a-768e05a8b871 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:33.509960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ae6012-69da-4251-b85a-768e05a8b871 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:33.511415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 497/1006] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:00:33.516873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e413a-2976-4a4e-a58b-a2a7328624b4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:33.517151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e413a-2976-4a4e-a58b-a2a7328624b4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:33.517448 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0e413a-2976-4a4e-a58b-a2a7328624b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:33.536728 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-3a0e413a-2976-4a4e-a58b-a2a7328624b4 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:33.537961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3a0e413a-2976-4a4e-a58b-a2a7328624b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:33.611629 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5589eea-a613-4161-bca4-e49738da4d5d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:33.618002 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 498/1007] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:33 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:00:33.650451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dcc77-19a0-4ec6-a8c2-42010b963f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:33.651062 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dcc77-19a0-4ec6-a8c2-42010b963f58 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:33.651339 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dcc77-19a0-4ec6-a8c2-42010b963f58 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:33.651682 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2dcc77-19a0-4ec6-a8c2-42010b963f58 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:33.652865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dcc77-19a0-4ec6-a8c2-42010b963f58 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:33.653167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dcc77-19a0-4ec6-a8c2-42010b963f58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:33.654543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2dcc77-19a0-4ec6-a8c2-42010b963f58 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:33.659596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 510/1008] 10.222.0.65 () {56 vars in 1112 bytes} [Wed Aug 31 19:00:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:33.665045 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e413a-2976-4a4e-a58b-a2a7328624b4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:33.686935 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:33.771100 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0e413a-2976-4a4e-a58b-a2a7328624b4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:33.805311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cfce8-e61b-4953-8f1a-1a8834083af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:33.805918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cfce8-e61b-4953-8f1a-1a8834083af6 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:33.806197 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cfce8-e61b-4953-8f1a-1a8834083af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:33.806469 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5cfce8-e61b-4953-8f1a-1a8834083af6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:33.807133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5cfce8-e61b-4953-8f1a-1a8834083af6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:33.809010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5cfce8-e61b-4953-8f1a-1a8834083af6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:33.829616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 499/1009] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:00:33 2022] POST /identity/v3/endpoints => generated 334 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:00:33.885862 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dcc77-19a0-4ec6-a8c2-42010b963f58 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:33.953158 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:33.995452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2dcc77-19a0-4ec6-a8c2-42010b963f58 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:34.036577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d8629-a354-46dc-90a1-2fd0124633bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:34.037216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d8629-a354-46dc-90a1-2fd0124633bf None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:34.037489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d8629-a354-46dc-90a1-2fd0124633bf None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:34.037753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9d8629-a354-46dc-90a1-2fd0124633bf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:34.038318 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d8629-a354-46dc-90a1-2fd0124633bf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:34.039644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9d8629-a354-46dc-90a1-2fd0124633bf None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-4a9d8629-a354-46dc-90a1-2fd0124633bf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:34.048309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:34.049376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:34.049376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:34.049376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:34.049376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:34.049376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:34.049376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:34.049376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:34.049376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:34.049376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:34.049376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:34.049376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:34.049376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:34.049376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:34.049376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:34.049376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:34.049376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:34.050285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:34.050285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:34.050285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:34.050285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:34.050285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:34.050285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:34.050285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:34.050285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:34.050285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:34.050285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:34.050285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:34.050285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:34.050285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 31 19:00:34.050285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:34.050285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 511/1010] 10.222.0.65 () {56 vars in 1126 bytes} [Wed Aug 31 19:00:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:34.055438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cfce8-e61b-4953-8f1a-1a8834083af6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:34.142368 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:34.167010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5cfce8-e61b-4953-8f1a-1a8834083af6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:34.205170 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28b579-d54c-450b-860f-f6a704167f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:34.205814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28b579-d54c-450b-860f-f6a704167f90 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:34.206095 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28b579-d54c-450b-860f-f6a704167f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:34.206371 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b28b579-d54c-450b-860f-f6a704167f90 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:34.207061 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28b579-d54c-450b-860f-f6a704167f90 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:34.207369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28b579-d54c-450b-860f-f6a704167f90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:34.208796 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b28b579-d54c-450b-860f-f6a704167f90 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:34.217024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 500/1011] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 19:00:34 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:34.222249 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d8629-a354-46dc-90a1-2fd0124633bf None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:34.327585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9d8629-a354-46dc-90a1-2fd0124633bf None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:34.351443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:34.364542 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085fbf10-730a-4cec-b34e-0ec38454e311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:34.365155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fbf10-730a-4cec-b34e-0ec38454e311 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:34.365457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fbf10-730a-4cec-b34e-0ec38454e311 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:34.365728 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085fbf10-730a-4cec-b34e-0ec38454e311 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:34.366365 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fbf10-730a-4cec-b34e-0ec38454e311 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:34.366665 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fbf10-730a-4cec-b34e-0ec38454e311 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:34.368057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085fbf10-730a-4cec-b34e-0ec38454e311 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:34.379810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 512/1012] 10.222.0.65 () {56 vars in 1122 bytes} [Wed Aug 31 19:00:34 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:34.380824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28b579-d54c-450b-860f-f6a704167f90 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:34.479789 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b28b579-d54c-450b-860f-f6a704167f90 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:34.519574 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c247a-c520-4549-9a8c-abc7c830f63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:34.520196 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c247a-c520-4549-9a8c-abc7c830f63c None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:34.520475 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c247a-c520-4549-9a8c-abc7c830f63c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:34.520745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c247a-c520-4549-9a8c-abc7c830f63c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:34.521457 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c247a-c520-4549-9a8c-abc7c830f63c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:34.522845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272c247a-c520-4549-9a8c-abc7c830f63c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:34.536913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:34.553793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 501/1013] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:00:34 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:00:35.209470 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601bd40-8e4f-423a-90b3-addcf7886911 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:35.209888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601bd40-8e4f-423a-90b3-addcf7886911 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:35.210070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d601bd40-8e4f-423a-90b3-addcf7886911 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:35.213359 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 513/1014] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:35.217251 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace536d-ef87-441b-9dac-137053613c73 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:35.217548 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace536d-ef87-441b-9dac-137053613c73 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:35.217827 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ace536d-ef87-441b-9dac-137053613c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:35.235693 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9ace536d-ef87-441b-9dac-137053613c73 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:35.236866 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9ace536d-ef87-441b-9dac-137053613c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:35.317378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 502/1015] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:35 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:35.585104 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601bd40-8e4f-423a-90b3-addcf7886911 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:35.690090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d601bd40-8e4f-423a-90b3-addcf7886911 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:35.724306 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a9d910-b15d-4c03-bb38-ef22ba26930f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:35.724913 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9d910-b15d-4c03-bb38-ef22ba26930f None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:35.725213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9d910-b15d-4c03-bb38-ef22ba26930f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:35.725482 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a9d910-b15d-4c03-bb38-ef22ba26930f None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:35.726139 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9d910-b15d-4c03-bb38-ef22ba26930f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:35.727495 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a9d910-b15d-4c03-bb38-ef22ba26930f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:35.745215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 514/1016] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 19:00:35 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:00:35.855995 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfda0c-03ba-4760-999d-ab4a993b4aba None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:35.856512 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfda0c-03ba-4760-999d-ab4a993b4aba None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:35.856591 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfda0c-03ba-4760-999d-ab4a993b4aba None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:35.858031 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 503/1017] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:35.863810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d6cff-272d-43aa-9ac9-a9f0ba0e4e83 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:35.864119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d6cff-272d-43aa-9ac9-a9f0ba0e4e83 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:35.864355 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2d6cff-272d-43aa-9ac9-a9f0ba0e4e83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:35.885841 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6d2d6cff-272d-43aa-9ac9-a9f0ba0e4e83 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:35.887112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6d2d6cff-272d-43aa-9ac9-a9f0ba0e4e83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:35.933329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:35.967069 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 515/1018] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:36.241071 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfda0c-03ba-4760-999d-ab4a993b4aba None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:36.346228 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfda0c-03ba-4760-999d-ab4a993b4aba None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:36.372900 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:36.382795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1590e57b-2eb2-49d0-ac3d-ea3e06dcf2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:36.383432 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590e57b-2eb2-49d0-ac3d-ea3e06dcf2dc None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:36.383703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590e57b-2eb2-49d0-ac3d-ea3e06dcf2dc None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:36.383973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1590e57b-2eb2-49d0-ac3d-ea3e06dcf2dc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:36.394691 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1590e57b-2eb2-49d0-ac3d-ea3e06dcf2dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:00:36.399472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-1590e57b-2eb2-49d0-ac3d-ea3e06dcf2dc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:00:36.399472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:36.399472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:36.399472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:36.399472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:36.399472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:36.399472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:36.399472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:36.399472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:36.399472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:36.399472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:36.399472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:36.399472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:00:36.399472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:00:36.399472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:00:36.399472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:00:36.400417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:00:36.400417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:00:36.400417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:00:36.400417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:00:36.400417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:36.400417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:36.400417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:36.400417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:36.400417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:36.400417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:36.400417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:36.400417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:36.400417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:36.400417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:36.400417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:36.400417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:36.401308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:36.401308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:36.401308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:36.401308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:36.401308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:00:36.401308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:00:36.401308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:00:36.401308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:00:36.401308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:00:36.401308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:00:36.401308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:00:36.401308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:36.403302 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 504/1019] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 31 19:00:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:36.407091 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d6cff-272d-43aa-9ac9-a9f0ba0e4e83 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:36.451930 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5a668-d582-4e8c-a8cf-f298f725616c None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:36.452425 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5a668-d582-4e8c-a8cf-f298f725616c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:36.452714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf5a668-d582-4e8c-a8cf-f298f725616c None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:36.455402 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 505/1020] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:36.460329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0bbaa-a722-4d76-aa32-6cbed8f37fba None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:36.460607 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0bbaa-a722-4d76-aa32-6cbed8f37fba None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:36.460874 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0bbaa-a722-4d76-aa32-6cbed8f37fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:36.480291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-23b0bbaa-a722-4d76-aa32-6cbed8f37fba None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:36.481557 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-23b0bbaa-a722-4d76-aa32-6cbed8f37fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:36.511304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2d6cff-272d-43aa-9ac9-a9f0ba0e4e83 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:36.547751 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68e275-5197-4439-87e8-ef4d8cf6a404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:36.548345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68e275-5197-4439-87e8-ef4d8cf6a404 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:36.548633 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68e275-5197-4439-87e8-ef4d8cf6a404 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:36.548970 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 506/1021] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:36 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:00:36.553531 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a68e275-5197-4439-87e8-ef4d8cf6a404 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:36.554310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68e275-5197-4439-87e8-ef4d8cf6a404 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:36.554680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68e275-5197-4439-87e8-ef4d8cf6a404 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:36.556078 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a68e275-5197-4439-87e8-ef4d8cf6a404 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:36.564694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 516/1022] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 31 19:00:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:36.570164 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0bbaa-a722-4d76-aa32-6cbed8f37fba None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:36.668269 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0bbaa-a722-4d76-aa32-6cbed8f37fba None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:36.684414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:36.703651 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e4d63-97d1-4a9f-aea0-f889be62bc5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:36.704211 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e4d63-97d1-4a9f-aea0-f889be62bc5b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:36.704485 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e4d63-97d1-4a9f-aea0-f889be62bc5b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:36.704748 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e4d63-97d1-4a9f-aea0-f889be62bc5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:36.715663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e4d63-97d1-4a9f-aea0-f889be62bc5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:36.717089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e4d63-97d1-4a9f-aea0-f889be62bc5b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:36.731826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-cf3e4d63-97d1-4a9f-aea0-f889be62bc5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:36.731826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:36.731826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:36.731826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:36.731826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:36.731826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:36.731826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:36.731826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:36.731826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:36.731826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:36.731826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:36.731826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:36.731826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:36.731826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:36.731826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:36.731826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:36.732751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:36.734206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:36.734206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:36.734206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:36.737627 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 507/1023] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:36.738716 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68e275-5197-4439-87e8-ef4d8cf6a404 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:36.774727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:36.816763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e4d63-97d1-4a9f-aea0-f889be62bc5b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:36.845325 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a68e275-5197-4439-87e8-ef4d8cf6a404 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:36.880540 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ea6a2-b762-4e78-9c09-efacf2f4ffc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:36.881473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ea6a2-b762-4e78-9c09-efacf2f4ffc6 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:36.882003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ea6a2-b762-4e78-9c09-efacf2f4ffc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:36.882463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ea6a2-b762-4e78-9c09-efacf2f4ffc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:36.883318 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ea6a2-b762-4e78-9c09-efacf2f4ffc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:36.883809 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ea6a2-b762-4e78-9c09-efacf2f4ffc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:36.885347 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ea6a2-b762-4e78-9c09-efacf2f4ffc6 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:36.898138 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 517/1024] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:36.903986 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ea6a2-b762-4e78-9c09-efacf2f4ffc6 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:36.930597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e4d63-97d1-4a9f-aea0-f889be62bc5b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:36.969661 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e41c0-39ba-46cd-b784-e9c0de8dd67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:36.970327 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e41c0-39ba-46cd-b784-e9c0de8dd67a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:36.970609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e41c0-39ba-46cd-b784-e9c0de8dd67a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:36.971359 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7e41c0-39ba-46cd-b784-e9c0de8dd67a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:36.972732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e41c0-39ba-46cd-b784-e9c0de8dd67a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:36.975089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7e41c0-39ba-46cd-b784-e9c0de8dd67a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:36.977222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 508/1025] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:36.982294 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e41c0-39ba-46cd-b784-e9c0de8dd67a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:36.999543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:37.007774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ea6a2-b762-4e78-9c09-efacf2f4ffc6 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:37.021361 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:37.044507 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f847fa-4a71-4f60-b1f9-aa87a22969f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:37.045130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f847fa-4a71-4f60-b1f9-aa87a22969f0 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:37.045429 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f847fa-4a71-4f60-b1f9-aa87a22969f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:37.045691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f847fa-4a71-4f60-b1f9-aa87a22969f0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:37.052935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f847fa-4a71-4f60-b1f9-aa87a22969f0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:37.054390 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f847fa-4a71-4f60-b1f9-aa87a22969f0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:37.066223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-f2f847fa-4a71-4f60-b1f9-aa87a22969f0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:00:37.066223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:37.066223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:37.066223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:37.066223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:37.066223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:37.066223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:37.066223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:37.066223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:37.066223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:37.066223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:37.066223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:37.066223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:00:37.066223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:00:37.066223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:00:37.066223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:37.067180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:37.068137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:37.068137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:37.068137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:00:37.068137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:00:37.068137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:00:37.068137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:00:37.068137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:00:37.068137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:00:37.068137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:00:37.068137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:37.068137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 518/1026] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 19:00:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:37.072957 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f847fa-4a71-4f60-b1f9-aa87a22969f0 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:37.094443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7e41c0-39ba-46cd-b784-e9c0de8dd67a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:37.131420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d0e94-f985-4c97-a6f2-00c8d1a52f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:37.132025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d0e94-f985-4c97-a6f2-00c8d1a52f5e None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:37.132297 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d0e94-f985-4c97-a6f2-00c8d1a52f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:37.132566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d0e94-f985-4c97-a6f2-00c8d1a52f5e None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:37.133210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d0e94-f985-4c97-a6f2-00c8d1a52f5e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:37.134616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d0e94-f985-4c97-a6f2-00c8d1a52f5e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:37.165238 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 509/1027] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 19:00:36 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:00:37.202007 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f847fa-4a71-4f60-b1f9-aa87a22969f0 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:37.244070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a25eab-c603-467a-bcf4-f44283415456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:37.244610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a25eab-c603-467a-bcf4-f44283415456 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:37.244884 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a25eab-c603-467a-bcf4-f44283415456 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:37.245154 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a25eab-c603-467a-bcf4-f44283415456 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:37.246468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a25eab-c603-467a-bcf4-f44283415456 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:37.246780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a25eab-c603-467a-bcf4-f44283415456 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:37.248300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a25eab-c603-467a-bcf4-f44283415456 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:37.269646 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 519/1028] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 19:00:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:37.274532 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d0e94-f985-4c97-a6f2-00c8d1a52f5e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:37.370159 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:37.385542 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d0e94-f985-4c97-a6f2-00c8d1a52f5e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:37.420601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a1bca1-74b0-47d5-933c-0d40309324aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:37.421228 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1bca1-74b0-47d5-933c-0d40309324aa None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:37.421515 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1bca1-74b0-47d5-933c-0d40309324aa None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:37.421787 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a1bca1-74b0-47d5-933c-0d40309324aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:37.491374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1bca1-74b0-47d5-933c-0d40309324aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:37.492814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a1bca1-74b0-47d5-933c-0d40309324aa None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:37.580238 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:37.594466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 510/1029] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:00:37 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:00:37.599998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a25eab-c603-467a-bcf4-f44283415456 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:37.615552 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:37.676908 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47484a-0c28-492f-9f4f-87416e978bb0 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:37.677262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47484a-0c28-492f-9f4f-87416e978bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:37.677488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c47484a-0c28-492f-9f4f-87416e978bb0 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:37.678830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 511/1030] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:00:37.683865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265a391-b20d-4b21-a708-b6bb5b1e8e7a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:37.684138 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265a391-b20d-4b21-a708-b6bb5b1e8e7a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:37.684405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265a391-b20d-4b21-a708-b6bb5b1e8e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:37.702294 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f265a391-b20d-4b21-a708-b6bb5b1e8e7a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:37.703458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f265a391-b20d-4b21-a708-b6bb5b1e8e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:37.707513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a25eab-c603-467a-bcf4-f44283415456 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:37.744423 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b752a5c-1b77-44b7-a6e8-02bf74bab5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:37.745055 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b752a5c-1b77-44b7-a6e8-02bf74bab5fd None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:37.745362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b752a5c-1b77-44b7-a6e8-02bf74bab5fd None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:37.745632 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b752a5c-1b77-44b7-a6e8-02bf74bab5fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:37.756370 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b752a5c-1b77-44b7-a6e8-02bf74bab5fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:37.757846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b752a5c-1b77-44b7-a6e8-02bf74bab5fd None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:37.772976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-0b752a5c-1b77-44b7-a6e8-02bf74bab5fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:37.772976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:37.772976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:37.772976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:37.772976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:37.772976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:37.772976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:37.772976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:37.772976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:37.772976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:37.772976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:37.772976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:37.772976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:37.772976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:37.772976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:37.772976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:37.774003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:37.782433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:37.782433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:37.782433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:37.782433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 520/1032] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:37.782665 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 512/1032] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:00:37.786111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b752a5c-1b77-44b7-a6e8-02bf74bab5fd None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:37.848745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:37.887667 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b752a5c-1b77-44b7-a6e8-02bf74bab5fd None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:37.920641 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e40dd-f45a-46e0-a268-3a9283669808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:37.921277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e40dd-f45a-46e0-a268-3a9283669808 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:37.921554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e40dd-f45a-46e0-a268-3a9283669808 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:37.921818 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e40dd-f45a-46e0-a268-3a9283669808 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:37.922447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e40dd-f45a-46e0-a268-3a9283669808 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:37.922739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e40dd-f45a-46e0-a268-3a9283669808 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:37.924079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e40dd-f45a-46e0-a268-3a9283669808 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:37.940001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 521/1033] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:37.941247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265a391-b20d-4b21-a708-b6bb5b1e8e7a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:38.039113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265a391-b20d-4b21-a708-b6bb5b1e8e7a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:38.043702 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e40dd-f45a-46e0-a268-3a9283669808 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:38.058314 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:38.077896 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccce035-ee6f-4148-96a0-01fc7ca4544d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:38.078523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccce035-ee6f-4148-96a0-01fc7ca4544d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:38.078799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccce035-ee6f-4148-96a0-01fc7ca4544d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:38.079076 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccce035-ee6f-4148-96a0-01fc7ca4544d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:38.086084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccce035-ee6f-4148-96a0-01fc7ca4544d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:38.087511 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccce035-ee6f-4148-96a0-01fc7ca4544d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:38.099262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-7ccce035-ee6f-4148-96a0-01fc7ca4544d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:38.099262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:38.099262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:38.099262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:38.099262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:38.099262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:38.099262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:38.099262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:38.099262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:38.099262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:38.099262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:38.099262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:38.099262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:00:38.099262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:00:38.099262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:38.100171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:38.101072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:00:38.101072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:00:38.101072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:00:38.101072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:00:38.101072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:00:38.101072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:00:38.101072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:38.101072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:38.101072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 513/1034] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:38.105988 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccce035-ee6f-4148-96a0-01fc7ca4544d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:38.151974 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e40dd-f45a-46e0-a268-3a9283669808 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:38.177936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:38.190885 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c4348-c6bf-4908-a1ce-585a08336e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:38.191552 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c4348-c6bf-4908-a1ce-585a08336e49 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:38.191837 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c4348-c6bf-4908-a1ce-585a08336e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:38.192104 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584c4348-c6bf-4908-a1ce-585a08336e49 None admin] PATH_INFO: `/v3/services/image` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:38.192731 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c4348-c6bf-4908-a1ce-585a08336e49 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:38.194118 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584c4348-c6bf-4908-a1ce-585a08336e49 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-584c4348-c6bf-4908-a1ce-585a08336e49 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:38.203999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:38.205244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:38.205244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:38.205244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:38.205244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:38.205244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:38.205244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:38.205244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:38.205244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:38.205244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:38.205244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:38.205244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:38.205244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:38.205244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:38.205244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:38.205244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:38.205244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:38.206275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:38.206275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:38.206275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:38.206275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:38.206275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:38.206275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:38.206275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:38.206275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:38.206275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:38.206275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:38.206275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:38.206275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:38.206275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:00:38.206275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:38.206275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 522/1035] 10.222.0.65 () {56 vars in 1105 bytes} [Wed Aug 31 19:00:38 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:38.213624 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c4348-c6bf-4908-a1ce-585a08336e49 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:38.223696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccce035-ee6f-4148-96a0-01fc7ca4544d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:38.258657 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae522b91-9560-4cf3-a7cc-b37aea7ae72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:38.259260 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae522b91-9560-4cf3-a7cc-b37aea7ae72e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:38.259540 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae522b91-9560-4cf3-a7cc-b37aea7ae72e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:38.259814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae522b91-9560-4cf3-a7cc-b37aea7ae72e None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:38.260553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae522b91-9560-4cf3-a7cc-b37aea7ae72e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:38.261179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae522b91-9560-4cf3-a7cc-b37aea7ae72e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:38.262862 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae522b91-9560-4cf3-a7cc-b37aea7ae72e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:38.275192 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 514/1036] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:00:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:38.280987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae522b91-9560-4cf3-a7cc-b37aea7ae72e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:38.323340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:38.328337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584c4348-c6bf-4908-a1ce-585a08336e49 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:38.363952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fba2a-7db1-413f-88d9-f71427bf63ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:38.364484 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fba2a-7db1-413f-88d9-f71427bf63ba None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:38.364879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fba2a-7db1-413f-88d9-f71427bf63ba None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:38.365153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9fba2a-7db1-413f-88d9-f71427bf63ba None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:38.365804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fba2a-7db1-413f-88d9-f71427bf63ba None admin] RBAC: Adding query filter params (name=image) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:38.366105 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fba2a-7db1-413f-88d9-f71427bf63ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:38.367567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9fba2a-7db1-413f-88d9-f71427bf63ba None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:38.375075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 523/1037] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:38 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:38.381131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fba2a-7db1-413f-88d9-f71427bf63ba None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:38.388839 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae522b91-9560-4cf3-a7cc-b37aea7ae72e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:38.425117 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9c9c9-d58d-490a-9328-2b3bd0d8df9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:38.425732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9c9c9-d58d-490a-9328-2b3bd0d8df9e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:38.426398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9c9c9-d58d-490a-9328-2b3bd0d8df9e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:38.426671 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a9c9c9-d58d-490a-9328-2b3bd0d8df9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:38.427387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9c9c9-d58d-490a-9328-2b3bd0d8df9e None admin] RBAC: Adding query filter params (role.id=f37b082e86f54a6a936cf9a83fe956a7, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=673a72ac9a9847e3812007b66ac7a5ea) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:38.427680 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9c9c9-d58d-490a-9328-2b3bd0d8df9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:38.429737 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a9c9c9-d58d-490a-9328-2b3bd0d8df9e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:38.436143 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 515/1038] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:00:38 2022] GET /identity/v3/role_assignments?user.id=673a72ac9a9847e3812007b66ac7a5ea&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=f37b082e86f54a6a936cf9a83fe956a7 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:38.440237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:38.494805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9fba2a-7db1-413f-88d9-f71427bf63ba None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:38.532261 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654b9b9-5659-4ccd-a8f2-4c53492ae3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:38.532840 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654b9b9-5659-4ccd-a8f2-4c53492ae3fc None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:38.533146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654b9b9-5659-4ccd-a8f2-4c53492ae3fc None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:38.533547 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d654b9b9-5659-4ccd-a8f2-4c53492ae3fc None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:38.534345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654b9b9-5659-4ccd-a8f2-4c53492ae3fc None admin] RBAC: Adding query filter params (type=image) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:38.534557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654b9b9-5659-4ccd-a8f2-4c53492ae3fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:38.536076 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d654b9b9-5659-4ccd-a8f2-4c53492ae3fc None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:38.545624 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 524/1039] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:38.547815 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9c9c9-d58d-490a-9328-2b3bd0d8df9e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:38.654285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a9c9c9-d58d-490a-9328-2b3bd0d8df9e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:38.655856 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:38.690336 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1cc2bf-dab6-425d-a071-7866993bf50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:38.690936 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cc2bf-dab6-425d-a071-7866993bf50c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:38.691214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cc2bf-dab6-425d-a071-7866993bf50c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:38.691479 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1cc2bf-dab6-425d-a071-7866993bf50c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:38.692167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1cc2bf-dab6-425d-a071-7866993bf50c None admin] RBAC: Adding query filter params (interface=public, service_id=32876a27d1ea47529033c109f8e766b5, region_id=RegionOne) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:38.692466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1cc2bf-dab6-425d-a071-7866993bf50c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:38.693858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1cc2bf-dab6-425d-a071-7866993bf50c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:38.703344 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 516/1040] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 31 19:00:38 2022] GET /identity/v3/endpoints?service_id=32876a27d1ea47529033c109f8e766b5&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:38.704508 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654b9b9-5659-4ccd-a8f2-4c53492ae3fc None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:38.805315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d654b9b9-5659-4ccd-a8f2-4c53492ae3fc None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:38.822319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:38.840966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae63ffb-06e6-4520-adc8-3c8c186476f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:38.841596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae63ffb-06e6-4520-adc8-3c8c186476f2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:38.841870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae63ffb-06e6-4520-adc8-3c8c186476f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:38.842134 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae63ffb-06e6-4520-adc8-3c8c186476f2 None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:38.842724 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae63ffb-06e6-4520-adc8-3c8c186476f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:38.844071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae63ffb-06e6-4520-adc8-3c8c186476f2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:38.857222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 525/1041] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 19:00:38 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:00:39.122494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5c448-ed6e-41a9-8591-0a17391f40b4 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:39.123055 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5c448-ed6e-41a9-8591-0a17391f40b4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:39.123055 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e5c448-ed6e-41a9-8591-0a17391f40b4 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:39.124842 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 517/1042] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:39.129943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335596ec-ba4d-455d-9c6a-d5ff1704dd04 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:39.130223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335596ec-ba4d-455d-9c6a-d5ff1704dd04 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:39.130492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335596ec-ba4d-455d-9c6a-d5ff1704dd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:39.148487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-335596ec-ba4d-455d-9c6a-d5ff1704dd04 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:39.149679 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-335596ec-ba4d-455d-9c6a-d5ff1704dd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:39.221389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 526/1043] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:39 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:39.493672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5c448-ed6e-41a9-8591-0a17391f40b4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:39.601342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e5c448-ed6e-41a9-8591-0a17391f40b4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:39.634272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f8b8b-242e-4f2a-9c6c-e24cd50d8aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:39.635055 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f8b8b-242e-4f2a-9c6c-e24cd50d8aab None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:39.635468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f8b8b-242e-4f2a-9c6c-e24cd50d8aab None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:39.635912 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f8b8b-242e-4f2a-9c6c-e24cd50d8aab None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:39.637679 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f8b8b-242e-4f2a-9c6c-e24cd50d8aab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:39.639214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6f8b8b-242e-4f2a-9c6c-e24cd50d8aab None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:39.640945 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 518/1044] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:39.646745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335596ec-ba4d-455d-9c6a-d5ff1704dd04 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:39.748704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335596ec-ba4d-455d-9c6a-d5ff1704dd04 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:39.759743 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:39.784820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab9ef98-e5b9-45aa-8377-27fd32cd53d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:39.785441 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9ef98-e5b9-45aa-8377-27fd32cd53d0 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:39.785721 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9ef98-e5b9-45aa-8377-27fd32cd53d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:39.785991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab9ef98-e5b9-45aa-8377-27fd32cd53d0 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:39.786630 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9ef98-e5b9-45aa-8377-27fd32cd53d0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:39.788051 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab9ef98-e5b9-45aa-8377-27fd32cd53d0 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:39.817218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 527/1045] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 19:00:39 2022] POST /identity/v3/users => generated 285 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:00:40.326910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c4424-6c55-40c8-924c-3c0067a974ce None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:40.327518 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c4424-6c55-40c8-924c-3c0067a974ce None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:40.327518 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c4424-6c55-40c8-924c-3c0067a974ce None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:40.328811 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 519/1046] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:40.334019 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e3b51-4a9d-44b4-9532-0b1a42c6ca87 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:40.334373 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e3b51-4a9d-44b4-9532-0b1a42c6ca87 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:40.334715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e3b51-4a9d-44b4-9532-0b1a42c6ca87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:40.353101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-cb9e3b51-4a9d-44b4-9532-0b1a42c6ca87 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:40.354321 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cb9e3b51-4a9d-44b4-9532-0b1a42c6ca87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:40.429236 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 528/1047] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:40.689611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c4424-6c55-40c8-924c-3c0067a974ce None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:40.725667 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:40.757762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7520b-8d8b-42b6-80d9-e669f550365d None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:40.758270 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7520b-8d8b-42b6-80d9-e669f550365d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:40.758348 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c7520b-8d8b-42b6-80d9-e669f550365d None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:40.759785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 529/1048] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:40.764412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6750fb5-709d-4e69-86c1-40867f2f0a7f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:40.764697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6750fb5-709d-4e69-86c1-40867f2f0a7f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:40.764969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6750fb5-709d-4e69-86c1-40867f2f0a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:40.784854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f6750fb5-709d-4e69-86c1-40867f2f0a7f None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:40.786071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f6750fb5-709d-4e69-86c1-40867f2f0a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:40.797740 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c4424-6c55-40c8-924c-3c0067a974ce None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:40.834893 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4795b9-4c98-46d9-b5a1-754c466dee9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:40.835519 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4795b9-4c98-46d9-b5a1-754c466dee9a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:40.835799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4795b9-4c98-46d9-b5a1-754c466dee9a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:40.836106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4795b9-4c98-46d9-b5a1-754c466dee9a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:40.847563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4795b9-4c98-46d9-b5a1-754c466dee9a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:00:40.852348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-ce4795b9-4c98-46d9-b5a1-754c466dee9a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:00:40.852348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:40.852348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:40.852348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:40.852348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:40.852348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:40.852348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:40.852348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:40.852348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:40.852348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:40.852348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:40.852348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:40.852348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:00:40.852348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:00:40.852348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:00:40.852348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:00:40.853333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:00:40.853333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:00:40.853333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:00:40.853333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:00:40.853333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:40.853333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:40.853333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:40.853333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:40.853333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:40.853333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:40.853333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:40.853333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:40.853333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:40.853333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:40.853333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:40.853333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:40.854194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:40.854194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:40.854194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:40.854194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:40.854194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:00:40.854194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:00:40.854194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:00:40.854194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:00:40.854194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:00:40.854194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:00:40.854194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:00:40.854194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:40.854194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 520/1049] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 31 19:00:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:40.859174 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4795b9-4c98-46d9-b5a1-754c466dee9a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:40.860724 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 530/1050] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:00:40.960735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:40.969678 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4795b9-4c98-46d9-b5a1-754c466dee9a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:41.004349 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecbc4c-74f8-4d36-a97d-5750e2c7cc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:41.004951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecbc4c-74f8-4d36-a97d-5750e2c7cc2c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:41.005252 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecbc4c-74f8-4d36-a97d-5750e2c7cc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:41.005527 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecbc4c-74f8-4d36-a97d-5750e2c7cc2c None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:41.006183 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecbc4c-74f8-4d36-a97d-5750e2c7cc2c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:41.006482 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecbc4c-74f8-4d36-a97d-5750e2c7cc2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:41.007837 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecbc4c-74f8-4d36-a97d-5750e2c7cc2c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:41.016485 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 521/1051] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 31 19:00:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:41.021976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6750fb5-709d-4e69-86c1-40867f2f0a7f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:41.050137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:41.126324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecbc4c-74f8-4d36-a97d-5750e2c7cc2c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:41.127068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6750fb5-709d-4e69-86c1-40867f2f0a7f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:41.165271 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b8d94-c74a-4899-8c13-c47fa2109167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:41.165905 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b8d94-c74a-4899-8c13-c47fa2109167 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:41.166184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b8d94-c74a-4899-8c13-c47fa2109167 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:41.166454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b8d94-c74a-4899-8c13-c47fa2109167 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:41.179145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b8d94-c74a-4899-8c13-c47fa2109167 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:41.181259 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b8d94-c74a-4899-8c13-c47fa2109167 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:41.197604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-2a9b8d94-c74a-4899-8c13-c47fa2109167 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:41.197604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:41.197604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:41.197604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:41.197604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:41.197604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:41.197604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:41.197604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:41.197604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:41.197604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:41.197604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:41.197604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:41.197604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:41.197604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:41.197604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:41.197604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:41.198549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:41.200008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:41.200008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:41.200008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:41.200008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 531/1052] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:41.204838 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b8d94-c74a-4899-8c13-c47fa2109167 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:41.253281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecbc4c-74f8-4d36-a97d-5750e2c7cc2c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:41.289309 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347a324-08a0-4744-ab5e-088acb5c2a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:41.289945 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347a324-08a0-4744-ab5e-088acb5c2a44 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:41.290224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347a324-08a0-4744-ab5e-088acb5c2a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:41.290498 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1347a324-08a0-4744-ab5e-088acb5c2a44 None admin] PATH_INFO: `/v3/services/image` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:41.290900 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:41.291160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347a324-08a0-4744-ab5e-088acb5c2a44 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:41.292512 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1347a324-08a0-4744-ab5e-088acb5c2a44 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-1347a324-08a0-4744-ab5e-088acb5c2a44 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:41.302674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:41.303670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:41.303670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:41.303670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:41.303670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:41.303670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:41.303670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:41.303670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:00:41.303670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:00:41.303670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:00:41.303670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:00:41.303670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:41.303670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:41.303670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:41.303670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:41.303670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:41.303670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:41.304526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:41.304526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:41.304526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:41.304526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:41.304526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:41.304526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:41.304526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:41.304526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:41.304526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:41.304526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:41.304526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:00:41.304526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:00:41.304526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 31 19:00:41.304526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:41.304526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 522/1053] 10.222.0.65 () {56 vars in 1105 bytes} [Wed Aug 31 19:00:41 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:41.310123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347a324-08a0-4744-ab5e-088acb5c2a44 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:41.318223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b8d94-c74a-4899-8c13-c47fa2109167 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:41.333465 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:41.355580 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359edc1-d9f6-4296-9e7f-fb5a25b8907e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:41.356192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359edc1-d9f6-4296-9e7f-fb5a25b8907e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:41.356469 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359edc1-d9f6-4296-9e7f-fb5a25b8907e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:41.356742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5359edc1-d9f6-4296-9e7f-fb5a25b8907e None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:41.357408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359edc1-d9f6-4296-9e7f-fb5a25b8907e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:41.357713 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359edc1-d9f6-4296-9e7f-fb5a25b8907e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:41.359069 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5359edc1-d9f6-4296-9e7f-fb5a25b8907e None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:41.377486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 532/1054] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:41.378662 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359edc1-d9f6-4296-9e7f-fb5a25b8907e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:41.419057 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1347a324-08a0-4744-ab5e-088acb5c2a44 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:41.455786 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77882e2a-e159-4bd4-918e-2065021e5750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:41.456457 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77882e2a-e159-4bd4-918e-2065021e5750 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:41.456743 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77882e2a-e159-4bd4-918e-2065021e5750 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:41.457010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77882e2a-e159-4bd4-918e-2065021e5750 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:41.457671 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77882e2a-e159-4bd4-918e-2065021e5750 None admin] RBAC: Adding query filter params (name=image) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:41.457972 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77882e2a-e159-4bd4-918e-2065021e5750 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:41.459826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77882e2a-e159-4bd4-918e-2065021e5750 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:41.471673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 523/1055] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:41.473520 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77882e2a-e159-4bd4-918e-2065021e5750 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:41.484889 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5359edc1-d9f6-4296-9e7f-fb5a25b8907e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:41.521903 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765ece4-3498-4b57-86c3-d4edea868b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:41.522657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765ece4-3498-4b57-86c3-d4edea868b60 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:41.522772 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765ece4-3498-4b57-86c3-d4edea868b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:41.522957 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f765ece4-3498-4b57-86c3-d4edea868b60 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:41.530498 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765ece4-3498-4b57-86c3-d4edea868b60 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:41.531917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f765ece4-3498-4b57-86c3-d4edea868b60 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:41.534562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:41.546769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-f765ece4-3498-4b57-86c3-d4edea868b60 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:00:41.546769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:41.546769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:41.546769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:41.546769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:41.546769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:41.546769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:41.546769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:41.546769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:41.546769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:41.546769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:41.546769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:41.546769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:00:41.546769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:00:41.546769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:00:41.546769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:41.547735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:41.548782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:41.548782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:41.548782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:00:41.548782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:00:41.548782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:00:41.548782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:00:41.548782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:00:41.548782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:00:41.548782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:00:41.548782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:41.548782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 533/1056] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 19:00:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:41.553502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765ece4-3498-4b57-86c3-d4edea868b60 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:41.571101 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:41.589411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77882e2a-e159-4bd4-918e-2065021e5750 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:41.628389 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f6373-a6bd-4aa2-bd44-bf527b5477bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:41.628389 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f6373-a6bd-4aa2-bd44-bf527b5477bb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:41.628693 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f6373-a6bd-4aa2-bd44-bf527b5477bb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:41.628899 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f6373-a6bd-4aa2-bd44-bf527b5477bb None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:41.629554 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f6373-a6bd-4aa2-bd44-bf527b5477bb None admin] RBAC: Adding query filter params (type=image) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:41.629856 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f6373-a6bd-4aa2-bd44-bf527b5477bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:41.631201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f6373-a6bd-4aa2-bd44-bf527b5477bb None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:41.636832 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 524/1057] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:41.642251 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f6373-a6bd-4aa2-bd44-bf527b5477bb None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:41.659981 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f765ece4-3498-4b57-86c3-d4edea868b60 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:41.695643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c59b2b-c2df-4bbf-9350-33b76ff3011f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:41.696250 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c59b2b-c2df-4bbf-9350-33b76ff3011f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:41.696529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c59b2b-c2df-4bbf-9350-33b76ff3011f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:41.696799 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c59b2b-c2df-4bbf-9350-33b76ff3011f None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:41.697631 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c59b2b-c2df-4bbf-9350-33b76ff3011f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:41.697933 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c59b2b-c2df-4bbf-9350-33b76ff3011f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:41.699397 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c59b2b-c2df-4bbf-9350-33b76ff3011f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:41.722951 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 534/1058] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 19:00:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:41.724317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c59b2b-c2df-4bbf-9350-33b76ff3011f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:41.748946 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f6373-a6bd-4aa2-bd44-bf527b5477bb None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:41.784412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeafcc97-d08d-46a4-9dcc-c498998bc087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:41.784935 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafcc97-d08d-46a4-9dcc-c498998bc087 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:41.785225 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafcc97-d08d-46a4-9dcc-c498998bc087 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:41.785494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeafcc97-d08d-46a4-9dcc-c498998bc087 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:41.786142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeafcc97-d08d-46a4-9dcc-c498998bc087 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:41.787248 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:41.787709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeafcc97-d08d-46a4-9dcc-c498998bc087 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:41.807439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 525/1059] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:00:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:00:41.809528 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5a416-7ac6-4037-9124-f1f44e958b03 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:41.809813 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5a416-7ac6-4037-9124-f1f44e958b03 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:41.810088 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a5a416-7ac6-4037-9124-f1f44e958b03 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:41.811328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 526/1060] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:41.816957 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f25d95-353c-4511-9c8d-1a4fb6a13de4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:41.817324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f25d95-353c-4511-9c8d-1a4fb6a13de4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:41.817545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f25d95-353c-4511-9c8d-1a4fb6a13de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:41.824976 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:41.838261 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-09f25d95-353c-4511-9c8d-1a4fb6a13de4 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:41.839455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-09f25d95-353c-4511-9c8d-1a4fb6a13de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:41.847008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c59b2b-c2df-4bbf-9350-33b76ff3011f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:41.894372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6ad6e-7754-47e4-b9a9-2b9aaeab5893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:41.895009 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6ad6e-7754-47e4-b9a9-2b9aaeab5893 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:41.895283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6ad6e-7754-47e4-b9a9-2b9aaeab5893 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:41.895551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b6ad6e-7754-47e4-b9a9-2b9aaeab5893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:41.906362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6ad6e-7754-47e4-b9a9-2b9aaeab5893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:41.907826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b6ad6e-7754-47e4-b9a9-2b9aaeab5893 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:41.918724 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 527/1061] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:00:41.929435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-d5b6ad6e-7754-47e4-b9a9-2b9aaeab5893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:41.929435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:41.929435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:41.929435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:41.929435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:41.929435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:41.929435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:41.929435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:41.929435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:41.929435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:41.929435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:41.929435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:41.929435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:41.929435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:41.929435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:41.929435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:41.930445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:41.931357 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:41.931357 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:41.931357 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:41.931357 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 535/1062] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:41.931599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f25d95-353c-4511-9c8d-1a4fb6a13de4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:42.038668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f25d95-353c-4511-9c8d-1a4fb6a13de4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:42.072463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca5b69-97df-419b-9330-bebc78aecf39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:42.073069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca5b69-97df-419b-9330-bebc78aecf39 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:42.073377 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca5b69-97df-419b-9330-bebc78aecf39 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:42.073652 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca5b69-97df-419b-9330-bebc78aecf39 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:42.074326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca5b69-97df-419b-9330-bebc78aecf39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:42.074626 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca5b69-97df-419b-9330-bebc78aecf39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:42.075994 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ca5b69-97df-419b-9330-bebc78aecf39 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:42.077438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:42.088603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 528/1063] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:42.094639 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6ad6e-7754-47e4-b9a9-2b9aaeab5893 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:42.191198 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca5b69-97df-419b-9330-bebc78aecf39 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:42.200178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b6ad6e-7754-47e4-b9a9-2b9aaeab5893 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:42.203331 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:42.239564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c494643-7426-4716-827e-4ed35f938aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:42.240531 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c494643-7426-4716-827e-4ed35f938aed None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:42.240989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c494643-7426-4716-827e-4ed35f938aed None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:42.241474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c494643-7426-4716-827e-4ed35f938aed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:42.250410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c494643-7426-4716-827e-4ed35f938aed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:42.252129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c494643-7426-4716-827e-4ed35f938aed None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:42.264012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-9c494643-7426-4716-827e-4ed35f938aed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:42.264012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:42.264012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:42.264012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:42.264012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:42.264012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:42.264012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:42.264012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:42.264012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:42.264012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:42.264012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:42.264012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:42.264012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:00:42.264012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:00:42.264012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:42.264994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:42.265938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:00:42.265938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:00:42.265938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:00:42.265938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:00:42.265938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:00:42.265938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:00:42.265938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:42.265938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:42.267930 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 536/1064] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:42.273348 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c494643-7426-4716-827e-4ed35f938aed None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:42.315026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ca5b69-97df-419b-9330-bebc78aecf39 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:42.351355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05596baa-497a-401d-8108-635e38c1d7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:42.352041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05596baa-497a-401d-8108-635e38c1d7dc None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:42.352268 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05596baa-497a-401d-8108-635e38c1d7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:42.352952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05596baa-497a-401d-8108-635e38c1d7dc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:42.365453 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05596baa-497a-401d-8108-635e38c1d7dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:00:42.370180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-05596baa-497a-401d-8108-635e38c1d7dc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:00:42.370180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:42.370180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:42.370180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:42.370180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:42.370180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:42.370180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:42.370180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:42.370180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:42.370180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:42.370180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:42.370180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:42.370180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:00:42.370180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:00:42.370180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:00:42.370180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:00:42.371131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:00:42.371131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:00:42.371131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:00:42.371131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:00:42.371131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:42.371131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:42.371131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:42.371131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:42.371131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:42.371131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:42.371131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:42.371131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:42.371131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:42.371131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:42.371131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:42.371131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:42.371991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:42.371991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:42.371991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:42.371991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:42.371991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:00:42.371991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:00:42.371991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:00:42.371991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:00:42.371991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:00:42.371991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:00:42.371991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:00:42.371991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:42.371991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 529/1065] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:00:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:42.378230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05596baa-497a-401d-8108-635e38c1d7dc None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:42.383536 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c494643-7426-4716-827e-4ed35f938aed None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:42.423402 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4c830-e7f9-4b7c-87bc-3cc666130496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:42.424057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4c830-e7f9-4b7c-87bc-3cc666130496 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:42.424445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4c830-e7f9-4b7c-87bc-3cc666130496 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:42.424758 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c4c830-e7f9-4b7c-87bc-3cc666130496 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:42.425557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4c830-e7f9-4b7c-87bc-3cc666130496 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:42.425860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4c830-e7f9-4b7c-87bc-3cc666130496 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:42.427407 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c4c830-e7f9-4b7c-87bc-3cc666130496 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:42.436630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:42.446782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 537/1066] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:00:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:42.448017 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4c830-e7f9-4b7c-87bc-3cc666130496 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:42.475216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:42.492675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05596baa-497a-401d-8108-635e38c1d7dc None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:42.529005 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59791c74-74fb-4f92-a819-30ad5a09e628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:42.529845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59791c74-74fb-4f92-a819-30ad5a09e628 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:42.530354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59791c74-74fb-4f92-a819-30ad5a09e628 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:42.530763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59791c74-74fb-4f92-a819-30ad5a09e628 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:42.531695 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59791c74-74fb-4f92-a819-30ad5a09e628 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:42.532137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59791c74-74fb-4f92-a819-30ad5a09e628 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:42.533672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59791c74-74fb-4f92-a819-30ad5a09e628 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:42.542957 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 530/1067] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:00:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:42.548591 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59791c74-74fb-4f92-a819-30ad5a09e628 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:42.555285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c4c830-e7f9-4b7c-87bc-3cc666130496 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:42.589617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af861c4-1d08-4151-bb8b-44ebf1bfe03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:42.590971 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af861c4-1d08-4151-bb8b-44ebf1bfe03f None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:42.591391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af861c4-1d08-4151-bb8b-44ebf1bfe03f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:42.591811 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af861c4-1d08-4151-bb8b-44ebf1bfe03f None admin] PATH_INFO: `/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/673a72ac9a9847e3812007b66ac7a5ea/roles/f37b082e86f54a6a936cf9a83fe956a7` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:42.594001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af861c4-1d08-4151-bb8b-44ebf1bfe03f None admin] RBAC: Authorizing `identity:create_grant(project_id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user_id=673a72ac9a9847e3812007b66ac7a5ea, role_id=f37b082e86f54a6a936cf9a83fe956a7)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:42.595415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af861c4-1d08-4151-bb8b-44ebf1bfe03f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:42.611939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 538/1068] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 19:00:42 2022] PUT /identity/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/673a72ac9a9847e3812007b66ac7a5ea/roles/f37b082e86f54a6a936cf9a83fe956a7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:00:42.656577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59791c74-74fb-4f92-a819-30ad5a09e628 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:42.675652 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:42.696326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de17fb-379e-4bb1-ae88-ce58efab52a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:42.696974 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de17fb-379e-4bb1-ae88-ce58efab52a7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:42.697279 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de17fb-379e-4bb1-ae88-ce58efab52a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:42.697556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5de17fb-379e-4bb1-ae88-ce58efab52a7 None admin] PATH_INFO: `/v3/users/aedb6f83de4c4567aa5b79c68cd88770` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:42.713080 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de17fb-379e-4bb1-ae88-ce58efab52a7 None admin] RBAC: Authorizing `identity:get_user(user_id=aedb6f83de4c4567aa5b79c68cd88770)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:42.715189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5de17fb-379e-4bb1-ae88-ce58efab52a7 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:42.717110 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 531/1069] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 19:00:42 2022] GET /identity/v3/users/aedb6f83de4c4567aa5b79c68cd88770 => generated 285 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:42.723014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af861c4-1d08-4151-bb8b-44ebf1bfe03f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:42.814588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:42.833500 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af861c4-1d08-4151-bb8b-44ebf1bfe03f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:42.869617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279843d7-0e51-427a-ab40-f01724d5e92c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:42.870141 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279843d7-0e51-427a-ab40-f01724d5e92c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:42.870415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279843d7-0e51-427a-ab40-f01724d5e92c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:42.870681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279843d7-0e51-427a-ab40-f01724d5e92c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:42.941388 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279843d7-0e51-427a-ab40-f01724d5e92c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:42.942839 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279843d7-0e51-427a-ab40-f01724d5e92c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:43.025963 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:43.048303 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 539/1070] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:00:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:00:43.049939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de17fb-379e-4bb1-ae88-ce58efab52a7 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:43.152607 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5de17fb-379e-4bb1-ae88-ce58efab52a7 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:43.187849 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb07078-59a3-4fb1-8224-3198cd439046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:43.188477 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb07078-59a3-4fb1-8224-3198cd439046 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:43.188759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb07078-59a3-4fb1-8224-3198cd439046 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:43.189032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb07078-59a3-4fb1-8224-3198cd439046 None admin] PATH_INFO: `/v3/projects/51958836cb824c6ab0e3ab66cd624f34` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:43.192646 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb07078-59a3-4fb1-8224-3198cd439046 None admin] RBAC: Authorizing `identity:get_project(project_id=51958836cb824c6ab0e3ab66cd624f34)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:43.192646 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb07078-59a3-4fb1-8224-3198cd439046 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:43.198287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 532/1071] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 19:00:43 2022] GET /identity/v3/projects/51958836cb824c6ab0e3ab66cd624f34 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:43.201207 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279843d7-0e51-427a-ab40-f01724d5e92c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:43.302447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279843d7-0e51-427a-ab40-f01724d5e92c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:43.336370 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f4ea1c5-0d85-4840-963b-3638793b6d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:43.336973 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ea1c5-0d85-4840-963b-3638793b6d21 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:43.337272 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ea1c5-0d85-4840-963b-3638793b6d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:43.337543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4ea1c5-0d85-4840-963b-3638793b6d21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:43.338258 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ea1c5-0d85-4840-963b-3638793b6d21 None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=51958836cb824c6ab0e3ab66cd624f34, user.id=aedb6f83de4c4567aa5b79c68cd88770) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:43.338562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ea1c5-0d85-4840-963b-3638793b6d21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:43.340084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f4ea1c5-0d85-4840-963b-3638793b6d21 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:43.345609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 540/1072] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:00:43 2022] GET /identity/v3/role_assignments?user.id=aedb6f83de4c4567aa5b79c68cd88770&scope.project.id=51958836cb824c6ab0e3ab66cd624f34&role.id=3580c894dd974d35bc95848b8ba22598 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:43.777695 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90de45c-4a10-4dc3-9d19-242887b5c432 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:43.778240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90de45c-4a10-4dc3-9d19-242887b5c432 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:43.778240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90de45c-4a10-4dc3-9d19-242887b5c432 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:43.779597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 533/1073] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:43.784629 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411af2ee-b91f-4219-a10d-379424ef1807 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:43.784974 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411af2ee-b91f-4219-a10d-379424ef1807 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:43.785340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411af2ee-b91f-4219-a10d-379424ef1807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:43.804007 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-411af2ee-b91f-4219-a10d-379424ef1807 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:43.805178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-411af2ee-b91f-4219-a10d-379424ef1807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:43.848967 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:43.881226 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 541/1074] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:00:44.143830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90de45c-4a10-4dc3-9d19-242887b5c432 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:44.195473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:44.256461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90de45c-4a10-4dc3-9d19-242887b5c432 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:44.292150 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f26dc4e-a4b7-496f-831f-76a6d00b69a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:44.292791 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26dc4e-a4b7-496f-831f-76a6d00b69a5 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:44.293078 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26dc4e-a4b7-496f-831f-76a6d00b69a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:44.293392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f26dc4e-a4b7-496f-831f-76a6d00b69a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:44.360980 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26dc4e-a4b7-496f-831f-76a6d00b69a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:44.362445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f26dc4e-a4b7-496f-831f-76a6d00b69a5 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:44.407478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:44.468587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 534/1075] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:00:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:00:44.474538 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411af2ee-b91f-4219-a10d-379424ef1807 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:44.574872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411af2ee-b91f-4219-a10d-379424ef1807 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:44.577039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a06f62-2fcf-4c99-bec5-03ffb94c92e2 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:44.577359 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a06f62-2fcf-4c99-bec5-03ffb94c92e2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:44.577635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a06f62-2fcf-4c99-bec5-03ffb94c92e2 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:44.579101 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 535/1076] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:00:44.583718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b0ed5-5207-4ad1-b306-18cf39efbfc6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:44.583995 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b0ed5-5207-4ad1-b306-18cf39efbfc6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:44.584265 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b0ed5-5207-4ad1-b306-18cf39efbfc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:44.601835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7b9b0ed5-5207-4ad1-b306-18cf39efbfc6 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:44.602997 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7b9b0ed5-5207-4ad1-b306-18cf39efbfc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:44.610231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fac37-bf85-4c77-98a0-c9528c97d5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:44.610860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fac37-bf85-4c77-98a0-c9528c97d5c5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:44.611141 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fac37-bf85-4c77-98a0-c9528c97d5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:44.611414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714fac37-bf85-4c77-98a0-c9528c97d5c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:44.621932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fac37-bf85-4c77-98a0-c9528c97d5c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:44.623364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714fac37-bf85-4c77-98a0-c9528c97d5c5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:44.638330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-714fac37-bf85-4c77-98a0-c9528c97d5c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:44.638330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:44.638330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:44.638330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:44.638330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:44.638330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:44.638330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:44.638330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:44.638330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:44.638330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:44.638330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:44.638330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:44.638330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:44.638330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:44.638330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:44.638330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:44.639376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:44.640281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:44.640281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:44.640281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:44.640281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 542/1077] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:44.645232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fac37-bf85-4c77-98a0-c9528c97d5c5 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:44.676818 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 536/1078] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:00:44.677018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:44.755731 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714fac37-bf85-4c77-98a0-c9528c97d5c5 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:44.789628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6473a-0229-4109-9433-f030e890ec4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:44.790232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6473a-0229-4109-9433-f030e890ec4d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:44.790508 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6473a-0229-4109-9433-f030e890ec4d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:44.790778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df6473a-0229-4109-9433-f030e890ec4d None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:44.791413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6473a-0229-4109-9433-f030e890ec4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:44.791713 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6473a-0229-4109-9433-f030e890ec4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:44.793610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df6473a-0229-4109-9433-f030e890ec4d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:44.813480 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 543/1079] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:44.946224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b0ed5-5207-4ad1-b306-18cf39efbfc6 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:45.021754 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:45.053665 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b0ed5-5207-4ad1-b306-18cf39efbfc6 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:45.087850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9bce34-af9a-4639-b726-289ad7d92500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:45.088470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bce34-af9a-4639-b726-289ad7d92500 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:45.088745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bce34-af9a-4639-b726-289ad7d92500 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:45.089007 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9bce34-af9a-4639-b726-289ad7d92500 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:45.099838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9bce34-af9a-4639-b726-289ad7d92500 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:00:45.104745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-0b9bce34-af9a-4639-b726-289ad7d92500 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:00:45.104745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:45.104745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:45.104745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:45.104745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:45.104745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:45.104745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:45.104745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:45.104745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:45.104745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:45.104745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:45.104745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:45.104745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:00:45.104745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:00:45.104745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:00:45.104745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:00:45.106355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:00:45.106355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:00:45.106355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:00:45.106355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:00:45.106355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:45.106355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:45.106355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:45.106355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:45.106355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:45.106355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:45.106355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:45.106355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:45.106355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:45.106355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:45.106355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:45.106355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:45.107184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:45.107184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:45.107184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:45.107184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:45.107184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:00:45.107184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:00:45.107184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:00:45.107184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:00:45.107184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:00:45.107184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:00:45.107184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 31 19:00:45.107184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:45.107184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 537/1080] 10.222.0.65 () {56 vars in 1120 bytes} [Wed Aug 31 19:00:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:45.112620 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6473a-0229-4109-9433-f030e890ec4d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:45.203629 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:45.213454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df6473a-0229-4109-9433-f030e890ec4d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:45.218948 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f08bd-f993-4fad-a8e8-404836f99013 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:45.219227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f08bd-f993-4fad-a8e8-404836f99013 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:45.219496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f08bd-f993-4fad-a8e8-404836f99013 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:45.220803 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 538/1081] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:00:45.225622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8363c-5953-4d8d-80aa-1b27231f358a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:45.225896 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8363c-5953-4d8d-80aa-1b27231f358a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:45.226165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8363c-5953-4d8d-80aa-1b27231f358a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:45.243964 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-86b8363c-5953-4d8d-80aa-1b27231f358a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:45.245118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-86b8363c-5953-4d8d-80aa-1b27231f358a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:45.248257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90075a5b-6f00-41f3-ae19-a5a0db8e22e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:45.248862 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90075a5b-6f00-41f3-ae19-a5a0db8e22e4 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:45.249138 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90075a5b-6f00-41f3-ae19-a5a0db8e22e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:45.249681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90075a5b-6f00-41f3-ae19-a5a0db8e22e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:45.250326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90075a5b-6f00-41f3-ae19-a5a0db8e22e4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:45.250622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90075a5b-6f00-41f3-ae19-a5a0db8e22e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:45.251960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90075a5b-6f00-41f3-ae19-a5a0db8e22e4 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:45.259683 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 544/1082] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 31 19:00:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:45.265121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90075a5b-6f00-41f3-ae19-a5a0db8e22e4 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:45.316059 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 539/1083] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:00:45.363662 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90075a5b-6f00-41f3-ae19-a5a0db8e22e4 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:45.396082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129378a-d53c-4fd9-815f-6e7e2057b948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:45.396695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129378a-d53c-4fd9-815f-6e7e2057b948 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:45.396969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129378a-d53c-4fd9-815f-6e7e2057b948 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:45.397253 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129378a-d53c-4fd9-815f-6e7e2057b948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:45.408167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129378a-d53c-4fd9-815f-6e7e2057b948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:45.409634 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129378a-d53c-4fd9-815f-6e7e2057b948 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:45.423319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-e129378a-d53c-4fd9-815f-6e7e2057b948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:45.423319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:45.423319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:45.423319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:45.423319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:45.423319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:45.423319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:45.423319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:45.423319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:45.423319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:45.423319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:45.423319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:45.423319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:45.423319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:45.423319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:45.423319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:45.424244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:45.425179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:45.425179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:45.425179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:45.425179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 545/1084] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:45.430416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8363c-5953-4d8d-80aa-1b27231f358a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:45.488686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:45.525861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8363c-5953-4d8d-80aa-1b27231f358a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:45.559539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f21c5d-bb5e-417a-860d-6ba40010bf2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:45.560126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f21c5d-bb5e-417a-860d-6ba40010bf2e None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:45.560397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f21c5d-bb5e-417a-860d-6ba40010bf2e None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:45.560660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f21c5d-bb5e-417a-860d-6ba40010bf2e None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:45.561312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f21c5d-bb5e-417a-860d-6ba40010bf2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:45.561605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f21c5d-bb5e-417a-860d-6ba40010bf2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:45.562954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f21c5d-bb5e-417a-860d-6ba40010bf2e None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:45.574397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 540/1085] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:45.580101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129378a-d53c-4fd9-815f-6e7e2057b948 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:45.584287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:45.584419 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f21c5d-bb5e-417a-860d-6ba40010bf2e None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:45.681242 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129378a-d53c-4fd9-815f-6e7e2057b948 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:45.684281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f21c5d-bb5e-417a-860d-6ba40010bf2e None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:45.706050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:45.714324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a675d91-2168-47af-b357-10b2e38fc825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:45.715622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a675d91-2168-47af-b357-10b2e38fc825 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:45.715898 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a675d91-2168-47af-b357-10b2e38fc825 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:45.716159 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a675d91-2168-47af-b357-10b2e38fc825 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:45.718761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15759d-6fdd-45e7-900e-86d60aacf912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:45.719375 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15759d-6fdd-45e7-900e-86d60aacf912 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:45.719648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15759d-6fdd-45e7-900e-86d60aacf912 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:45.719909 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15759d-6fdd-45e7-900e-86d60aacf912 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:45.723934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a675d91-2168-47af-b357-10b2e38fc825 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:45.725400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a675d91-2168-47af-b357-10b2e38fc825 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:45.729622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15759d-6fdd-45e7-900e-86d60aacf912 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:00:45.734913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-8a15759d-6fdd-45e7-900e-86d60aacf912 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:00:45.734913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:45.734913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:45.734913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:45.734913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:45.734913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:45.734913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:45.734913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:45.734913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:45.734913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:45.734913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:45.734913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:45.734913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:00:45.734913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:00:45.734913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:00:45.734913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:00:45.735942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:00:45.735942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:00:45.735942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:00:45.735942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:00:45.735942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:45.735942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:45.735942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:45.735942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:45.735942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:45.735942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:45.735942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:45.735942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:45.735942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:45.735942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:45.735942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:45.735942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:45.736773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:45.736773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:45.736773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:45.736773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:45.736773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:00:45.736773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:00:45.736773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:00:45.736773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:00:45.736773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:00:45.736773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:00:45.736773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:00:45.736773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:45.736773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 541/1086] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:00:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:45.737493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-0a675d91-2168-47af-b357-10b2e38fc825 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:00:45.737493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:45.737493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:45.737493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:45.737493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:45.737493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:45.737493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:45.737493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:45.737493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:45.737493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:45.737493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:45.737493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:45.737493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:00:45.737493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:00:45.737493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:00:45.737493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:45.738938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:45.740099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:45.740099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:45.740099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:00:45.740099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:00:45.740099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:00:45.740099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:00:45.740099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:00:45.740099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:00:45.740099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 31 19:00:45.740099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:45.740099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 546/1087] 10.222.0.65 () {56 vars in 1093 bytes} [Wed Aug 31 19:00:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:00:45.741846 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15759d-6fdd-45e7-900e-86d60aacf912 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:45.743237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a675d91-2168-47af-b357-10b2e38fc825 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:45.804027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:45.838562 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15759d-6fdd-45e7-900e-86d60aacf912 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:45.843011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a675d91-2168-47af-b357-10b2e38fc825 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:45.872104 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd12ed-2e98-43ef-ade3-45a579d7cc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:45.872692 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd12ed-2e98-43ef-ade3-45a579d7cc18 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:45.872966 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd12ed-2e98-43ef-ade3-45a579d7cc18 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:45.873255 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd12ed-2e98-43ef-ade3-45a579d7cc18 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:45.873900 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd12ed-2e98-43ef-ade3-45a579d7cc18 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:45.874445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd12ed-2e98-43ef-ade3-45a579d7cc18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:45.875788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd12ed-2e98-43ef-ade3-45a579d7cc18 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:45.876294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dc6b5-f5b5-4d26-b22c-2e9913147f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:45.876893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dc6b5-f5b5-4d26-b22c-2e9913147f07 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:45.877174 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dc6b5-f5b5-4d26-b22c-2e9913147f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:45.877503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dc6b5-f5b5-4d26-b22c-2e9913147f07 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:45.878234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7dc6b5-f5b5-4d26-b22c-2e9913147f07 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:45.878536 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7dc6b5-f5b5-4d26-b22c-2e9913147f07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:45.879936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7dc6b5-f5b5-4d26-b22c-2e9913147f07 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:45.884194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 542/1088] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:00:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:45.890672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd12ed-2e98-43ef-ade3-45a579d7cc18 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:45.904052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 547/1089] 10.222.0.65 () {56 vars in 1133 bytes} [Wed Aug 31 19:00:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:45.905117 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dc6b5-f5b5-4d26-b22c-2e9913147f07 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:45.936434 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:45.990082 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd12ed-2e98-43ef-ade3-45a579d7cc18 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:46.001834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dc6b5-f5b5-4d26-b22c-2e9913147f07 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:46.023761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b7aec-fb35-40df-b7fc-76f169bf2e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:46.024039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:46.024378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b7aec-fb35-40df-b7fc-76f169bf2e1c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:46.024663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b7aec-fb35-40df-b7fc-76f169bf2e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:46.024937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58b7aec-fb35-40df-b7fc-76f169bf2e1c None admin] PATH_INFO: `/v3/users/aedb6f83de4c4567aa5b79c68cd88770` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:46.026281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b7aec-fb35-40df-b7fc-76f169bf2e1c None admin] RBAC: Authorizing `identity:get_user(user_id=aedb6f83de4c4567aa5b79c68cd88770)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:46.027701 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58b7aec-fb35-40df-b7fc-76f169bf2e1c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:46.029321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 543/1090] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 19:00:45 2022] GET /identity/v3/users/aedb6f83de4c4567aa5b79c68cd88770 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:46.034909 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b7aec-fb35-40df-b7fc-76f169bf2e1c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:46.036463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa7fdd-ebb0-480a-bcb4-18d094ca6a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:46.037069 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa7fdd-ebb0-480a-bcb4-18d094ca6a74 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:46.037382 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa7fdd-ebb0-480a-bcb4-18d094ca6a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:46.037671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa7fdd-ebb0-480a-bcb4-18d094ca6a74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:46.104593 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa7fdd-ebb0-480a-bcb4-18d094ca6a74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:46.106541 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa7fdd-ebb0-480a-bcb4-18d094ca6a74 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:46.135975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58b7aec-fb35-40df-b7fc-76f169bf2e1c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:46.149153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:46.171462 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380e8f0-e318-4dd4-9669-60808c52ba39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:46.172082 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380e8f0-e318-4dd4-9669-60808c52ba39 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:46.172358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380e8f0-e318-4dd4-9669-60808c52ba39 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:46.172634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8380e8f0-e318-4dd4-9669-60808c52ba39 None admin] PATH_INFO: `/v3/projects/51958836cb824c6ab0e3ab66cd624f34` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:46.173760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380e8f0-e318-4dd4-9669-60808c52ba39 None admin] RBAC: Authorizing `identity:get_project(project_id=51958836cb824c6ab0e3ab66cd624f34)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:46.175168 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8380e8f0-e318-4dd4-9669-60808c52ba39 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:46.176779 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 544/1091] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 19:00:46 2022] GET /identity/v3/projects/51958836cb824c6ab0e3ab66cd624f34 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:46.182696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380e8f0-e318-4dd4-9669-60808c52ba39 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:46.205092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 548/1092] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:00:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 19:00:46.210850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa7fdd-ebb0-480a-bcb4-18d094ca6a74 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:46.234104 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:46.286266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8380e8f0-e318-4dd4-9669-60808c52ba39 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:46.309293 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa7fdd-ebb0-480a-bcb4-18d094ca6a74 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:46.320525 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c92cdd-166e-49b5-b029-298e2fe7fed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:46.321174 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c92cdd-166e-49b5-b029-298e2fe7fed4 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:46.321481 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c92cdd-166e-49b5-b029-298e2fe7fed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:46.321752 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c92cdd-166e-49b5-b029-298e2fe7fed4 None admin] PATH_INFO: `/v3/projects/51958836cb824c6ab0e3ab66cd624f34/users/aedb6f83de4c4567aa5b79c68cd88770/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:46.323741 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c92cdd-166e-49b5-b029-298e2fe7fed4 None admin] RBAC: Authorizing `identity:create_grant(project_id=51958836cb824c6ab0e3ab66cd624f34, user_id=aedb6f83de4c4567aa5b79c68cd88770, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:46.325215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c92cdd-166e-49b5-b029-298e2fe7fed4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:46.338262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 545/1093] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 19:00:46 2022] PUT /identity/v3/projects/51958836cb824c6ab0e3ab66cd624f34/users/aedb6f83de4c4567aa5b79c68cd88770/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:00:46.343067 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e403e-06b2-4c3f-8d97-4fc5054a35b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:46.343715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e403e-06b2-4c3f-8d97-4fc5054a35b2 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:46.343994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e403e-06b2-4c3f-8d97-4fc5054a35b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:46.344270 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e403e-06b2-4c3f-8d97-4fc5054a35b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:46.356661 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e403e-06b2-4c3f-8d97-4fc5054a35b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:46.358203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e403e-06b2-4c3f-8d97-4fc5054a35b2 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:46.375412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-e28e403e-06b2-4c3f-8d97-4fc5054a35b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:46.375412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:46.375412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:46.375412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:46.375412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:46.375412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:46.375412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:46.375412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:46.375412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:46.375412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:46.375412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:46.375412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:46.375412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:00:46.375412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:00:46.375412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:00:46.375412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:46.381656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:00:46.382652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:00:46.382652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:00:46.382652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:46.382652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 549/1094] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:46.382901 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c92cdd-166e-49b5-b029-298e2fe7fed4 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:46.426710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:46.488805 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c92cdd-166e-49b5-b029-298e2fe7fed4 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:46.523295 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28c3af-b9f7-4866-8dd2-e4bcab9f0755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:46.523891 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28c3af-b9f7-4866-8dd2-e4bcab9f0755 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:46.524170 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28c3af-b9f7-4866-8dd2-e4bcab9f0755 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:46.524441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf28c3af-b9f7-4866-8dd2-e4bcab9f0755 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:46.525068 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28c3af-b9f7-4866-8dd2-e4bcab9f0755 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:46.525392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28c3af-b9f7-4866-8dd2-e4bcab9f0755 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:46.526736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf28c3af-b9f7-4866-8dd2-e4bcab9f0755 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:46.538367 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 546/1095] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:00:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:46.544588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e403e-06b2-4c3f-8d97-4fc5054a35b2 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:46.618016 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:46.643216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e403e-06b2-4c3f-8d97-4fc5054a35b2 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:46.676639 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f826f-739b-4ac1-b4d1-1db971a8a5f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:46.677292 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f826f-739b-4ac1-b4d1-1db971a8a5f5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:46.677572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f826f-739b-4ac1-b4d1-1db971a8a5f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:46.677840 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f826f-739b-4ac1-b4d1-1db971a8a5f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:46.684274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f826f-739b-4ac1-b4d1-1db971a8a5f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:46.685729 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f826f-739b-4ac1-b4d1-1db971a8a5f5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:46.696619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-200f826f-739b-4ac1-b4d1-1db971a8a5f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:46.696619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:46.696619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:46.696619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:46.696619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:46.696619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:46.696619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:46.696619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:46.696619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:46.696619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:46.696619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:46.696619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:46.696619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:00:46.696619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:00:46.696619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:46.697694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:46.698609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:00:46.698609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:00:46.698609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:00:46.698609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:00:46.698609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:00:46.698609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:00:46.698609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:00:46.698609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:00:46.698609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 550/1096] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:00:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:46.703611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28c3af-b9f7-4866-8dd2-e4bcab9f0755 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:46.799692 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:46.802553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf28c3af-b9f7-4866-8dd2-e4bcab9f0755 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:46.835597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea14d549-fdea-4f64-85af-a5cd8736298d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:46.836191 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14d549-fdea-4f64-85af-a5cd8736298d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:46.836470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14d549-fdea-4f64-85af-a5cd8736298d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:46.836740 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea14d549-fdea-4f64-85af-a5cd8736298d None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:46.837458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14d549-fdea-4f64-85af-a5cd8736298d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:46.837742 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14d549-fdea-4f64-85af-a5cd8736298d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:46.839139 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea14d549-fdea-4f64-85af-a5cd8736298d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:46.851374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 547/1097] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:00:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:46.857450 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f826f-739b-4ac1-b4d1-1db971a8a5f5 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:46.955073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f826f-739b-4ac1-b4d1-1db971a8a5f5 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:46.987648 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c743e17-1094-4788-9cd0-606e60129bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:46.988345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c743e17-1094-4788-9cd0-606e60129bb6 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:46.988695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c743e17-1094-4788-9cd0-606e60129bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:46.989056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c743e17-1094-4788-9cd0-606e60129bb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:46.989909 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c743e17-1094-4788-9cd0-606e60129bb6 None admin] RBAC: Adding query filter params (role.id=f37b082e86f54a6a936cf9a83fe956a7, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=673a72ac9a9847e3812007b66ac7a5ea) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:46.990217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c743e17-1094-4788-9cd0-606e60129bb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:46.991630 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c743e17-1094-4788-9cd0-606e60129bb6 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:46.993404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:47.009480 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 551/1098] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:00:46 2022] GET /identity/v3/role_assignments?user.id=673a72ac9a9847e3812007b66ac7a5ea&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=f37b082e86f54a6a936cf9a83fe956a7 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:48.237292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236ce91-33b1-49a0-a86b-9a8721459385 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:48.237803 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236ce91-33b1-49a0-a86b-9a8721459385 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:48.237919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5236ce91-33b1-49a0-a86b-9a8721459385 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:48.239182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 548/1099] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:48.244636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d6317-d06b-472a-a051-d7c9c7f0209e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:48.245048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d6317-d06b-472a-a051-d7c9c7f0209e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:48.245172 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529d6317-d06b-472a-a051-d7c9c7f0209e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:48.263023 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-529d6317-d06b-472a-a051-d7c9c7f0209e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:48.264186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-529d6317-d06b-472a-a051-d7c9c7f0209e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:48.328003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 552/1100] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:48.599408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236ce91-33b1-49a0-a86b-9a8721459385 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:48.697020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5236ce91-33b1-49a0-a86b-9a8721459385 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:48.729721 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac787d-fc91-4075-be54-8afdbbd27393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:48.730457 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac787d-fc91-4075-be54-8afdbbd27393 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:48.730737 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac787d-fc91-4075-be54-8afdbbd27393 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:48.731007 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac787d-fc91-4075-be54-8afdbbd27393 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:48.740558 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac787d-fc91-4075-be54-8afdbbd27393 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:00:48.745270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-ceac787d-fc91-4075-be54-8afdbbd27393 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:00:48.745270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:00:48.745270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:00:48.745270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:00:48.745270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:00:48.745270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:00:48.745270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:00:48.745270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:00:48.745270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:00:48.745270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:00:48.745270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:00:48.745270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:00:48.745270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:00:48.745270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:00:48.745270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:00:48.745270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:00:48.747305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:00:48.747305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:00:48.747305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:00:48.747305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:00:48.747305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:00:48.747305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:00:48.747305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:00:48.747305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:00:48.747305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:00:48.747305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:00:48.747305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:00:48.747305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:00:48.747305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:00:48.747305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:00:48.747305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:00:48.747305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:00:48.748516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:00:48.748516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:00:48.748516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:00:48.748516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:00:48.748516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:00:48.748516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:00:48.748516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:00:48.748516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:00:48.748516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:00:48.748516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:00:48.748516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:00:48.748516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:00:48.748516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 549/1101] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:00:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:00:48.753091 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d6317-d06b-472a-a051-d7c9c7f0209e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:48.847741 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529d6317-d06b-472a-a051-d7c9c7f0209e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:48.872897 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:48.881034 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da9460-4011-48ac-87b3-eca4c4b20a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:48.881652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da9460-4011-48ac-87b3-eca4c4b20a5b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:48.881922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da9460-4011-48ac-87b3-eca4c4b20a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:48.882184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24da9460-4011-48ac-87b3-eca4c4b20a5b None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:48.882822 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da9460-4011-48ac-87b3-eca4c4b20a5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:48.883113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da9460-4011-48ac-87b3-eca4c4b20a5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:48.884447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24da9460-4011-48ac-87b3-eca4c4b20a5b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:48.892770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 553/1102] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:00:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:48.898403 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac787d-fc91-4075-be54-8afdbbd27393 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:48.919997 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:48.994831 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac787d-fc91-4075-be54-8afdbbd27393 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:49.028169 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a241291-9d35-48d4-abb6-e42a2ad59097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:49.028773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241291-9d35-48d4-abb6-e42a2ad59097 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:49.029053 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241291-9d35-48d4-abb6-e42a2ad59097 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:49.029353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241291-9d35-48d4-abb6-e42a2ad59097 None admin] PATH_INFO: `/v3/users/aedb6f83de4c4567aa5b79c68cd88770` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:49.030447 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a241291-9d35-48d4-abb6-e42a2ad59097 None admin] RBAC: Authorizing `identity:get_user(user_id=aedb6f83de4c4567aa5b79c68cd88770)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:49.031857 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a241291-9d35-48d4-abb6-e42a2ad59097 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:49.033433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 550/1103] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 19:00:48 2022] GET /identity/v3/users/aedb6f83de4c4567aa5b79c68cd88770 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:49.039319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da9460-4011-48ac-87b3-eca4c4b20a5b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:49.134554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24da9460-4011-48ac-87b3-eca4c4b20a5b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:49.165600 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0835ee-a6be-4546-aaa4-eef92aefbeb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:49.166176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0835ee-a6be-4546-aaa4-eef92aefbeb3 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:49.166447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0835ee-a6be-4546-aaa4-eef92aefbeb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:49.166714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0835ee-a6be-4546-aaa4-eef92aefbeb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:49.221403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:49.232225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0835ee-a6be-4546-aaa4-eef92aefbeb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:49.233693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0835ee-a6be-4546-aaa4-eef92aefbeb3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:49.327638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 554/1104] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:00:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:00:49.333454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a241291-9d35-48d4-abb6-e42a2ad59097 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:49.417557 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:49.429694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a241291-9d35-48d4-abb6-e42a2ad59097 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:49.464434 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37aa3e0-bdd1-4636-9702-c9c876cead59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:49.465592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37aa3e0-bdd1-4636-9702-c9c876cead59 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:49.465867 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37aa3e0-bdd1-4636-9702-c9c876cead59 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:49.466131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37aa3e0-bdd1-4636-9702-c9c876cead59 None admin] PATH_INFO: `/v3/projects/51958836cb824c6ab0e3ab66cd624f34` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:49.467225 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37aa3e0-bdd1-4636-9702-c9c876cead59 None admin] RBAC: Authorizing `identity:get_project(project_id=51958836cb824c6ab0e3ab66cd624f34)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:49.468638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37aa3e0-bdd1-4636-9702-c9c876cead59 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:49.470272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 551/1105] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 19:00:49 2022] GET /identity/v3/projects/51958836cb824c6ab0e3ab66cd624f34 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:49.476506 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0835ee-a6be-4546-aaa4-eef92aefbeb3 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:49.563002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:49.572545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0835ee-a6be-4546-aaa4-eef92aefbeb3 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:49.606790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e8927d-cb9c-4fcf-9b84-1b65c6ad7487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:49.607375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8927d-cb9c-4fcf-9b84-1b65c6ad7487 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:49.607652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8927d-cb9c-4fcf-9b84-1b65c6ad7487 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:49.607924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e8927d-cb9c-4fcf-9b84-1b65c6ad7487 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:49.608637 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8927d-cb9c-4fcf-9b84-1b65c6ad7487 None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=51958836cb824c6ab0e3ab66cd624f34, user.id=aedb6f83de4c4567aa5b79c68cd88770) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:49.608943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8927d-cb9c-4fcf-9b84-1b65c6ad7487 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:49.610378 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e8927d-cb9c-4fcf-9b84-1b65c6ad7487 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:49.615959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 555/1106] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:00:49 2022] GET /identity/v3/role_assignments?user.id=aedb6f83de4c4567aa5b79c68cd88770&scope.project.id=51958836cb824c6ab0e3ab66cd624f34&role.id=3580c894dd974d35bc95848b8ba22598 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:51.523288 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50f7ec-4af3-46cd-98df-46a6fa23e2a2 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:51.523806 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50f7ec-4af3-46cd-98df-46a6fa23e2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:51.523806 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e50f7ec-4af3-46cd-98df-46a6fa23e2a2 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:51.525136 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 552/1107] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:51.531098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3492f5-41c6-4681-835a-c95ed46d2c1a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:51.531495 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3492f5-41c6-4681-835a-c95ed46d2c1a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:51.531642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3492f5-41c6-4681-835a-c95ed46d2c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:51.549310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9c3492f5-41c6-4681-835a-c95ed46d2c1a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:51.550482 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9c3492f5-41c6-4681-835a-c95ed46d2c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:51.614450 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 556/1108] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:51.884895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50f7ec-4af3-46cd-98df-46a6fa23e2a2 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:51.983389 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e50f7ec-4af3-46cd-98df-46a6fa23e2a2 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:52.017490 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39263343-ae66-4ebe-a6a7-367e5bbef9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:52.018098 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39263343-ae66-4ebe-a6a7-367e5bbef9bb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:52.018372 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39263343-ae66-4ebe-a6a7-367e5bbef9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:52.018638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39263343-ae66-4ebe-a6a7-367e5bbef9bb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:52.019837 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39263343-ae66-4ebe-a6a7-367e5bbef9bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:52.021276 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39263343-ae66-4ebe-a6a7-367e5bbef9bb None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:52.022794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 553/1109] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:00:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:52.028522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3492f5-41c6-4681-835a-c95ed46d2c1a None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:52.108892 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:52.125207 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3492f5-41c6-4681-835a-c95ed46d2c1a None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:52.157552 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd34698-f199-49de-9d95-d0f18d7e401b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:52.158084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd34698-f199-49de-9d95-d0f18d7e401b None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:52.158359 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd34698-f199-49de-9d95-d0f18d7e401b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:52.158625 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd34698-f199-49de-9d95-d0f18d7e401b None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:52.159256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd34698-f199-49de-9d95-d0f18d7e401b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:52.160662 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd34698-f199-49de-9d95-d0f18d7e401b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:52.181317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 557/1110] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 19:00:52 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:00:54.110349 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774154ef-7b47-4a69-9bfe-16666fb36841 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:54.110881 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774154ef-7b47-4a69-9bfe-16666fb36841 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:54.110881 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774154ef-7b47-4a69-9bfe-16666fb36841 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:54.112210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 554/1111] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:54.117234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bab91-5049-4de4-a608-9c221a1e17eb None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:54.117631 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bab91-5049-4de4-a608-9c221a1e17eb None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:54.117789 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49bab91-5049-4de4-a608-9c221a1e17eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:54.136209 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c49bab91-5049-4de4-a608-9c221a1e17eb None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:54.137977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c49bab91-5049-4de4-a608-9c221a1e17eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:54.202278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 558/1112] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:54.470284 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774154ef-7b47-4a69-9bfe-16666fb36841 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:54.495034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:54.570150 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774154ef-7b47-4a69-9bfe-16666fb36841 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:54.602142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba903d7-f715-4568-b0a7-08767def97a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:54.602747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba903d7-f715-4568-b0a7-08767def97a6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:54.603016 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba903d7-f715-4568-b0a7-08767def97a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:54.603279 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba903d7-f715-4568-b0a7-08767def97a6 None admin] PATH_INFO: `/v3/roles/470b660efbe04083a803e79d8b647160` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:54.604624 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba903d7-f715-4568-b0a7-08767def97a6 None admin] RBAC: Authorizing `identity:get_role(role_id=470b660efbe04083a803e79d8b647160)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:54.606520 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba903d7-f715-4568-b0a7-08767def97a6 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:54.607802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 555/1113] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 19:00:54 2022] GET /identity/v3/roles/470b660efbe04083a803e79d8b647160 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:54.613516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bab91-5049-4de4-a608-9c221a1e17eb None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:54.680252 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:54.711642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49bab91-5049-4de4-a608-9c221a1e17eb None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:54.746050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189b7c8-e522-43c0-86f3-dbce0d539b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:54.746605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189b7c8-e522-43c0-86f3-dbce0d539b35 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:54.746883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189b7c8-e522-43c0-86f3-dbce0d539b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:54.747154 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189b7c8-e522-43c0-86f3-dbce0d539b35 None admin] PATH_INFO: `/v3/users/4d166a3a044642d489c3bdf5122d7613` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:54.760244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189b7c8-e522-43c0-86f3-dbce0d539b35 None admin] RBAC: Authorizing `identity:get_user(user_id=4d166a3a044642d489c3bdf5122d7613)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:54.761722 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f189b7c8-e522-43c0-86f3-dbce0d539b35 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:54.763307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 559/1114] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 19:00:54 2022] GET /identity/v3/users/4d166a3a044642d489c3bdf5122d7613 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:54.769056 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba903d7-f715-4568-b0a7-08767def97a6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:54.857433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:54.866218 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba903d7-f715-4568-b0a7-08767def97a6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:54.898660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111b843-fa8c-465e-8136-457d92349d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:54.899249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111b843-fa8c-465e-8136-457d92349d7d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:54.899528 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111b843-fa8c-465e-8136-457d92349d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:54.899798 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3111b843-fa8c-465e-8136-457d92349d7d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:54.966145 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111b843-fa8c-465e-8136-457d92349d7d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:54.967584 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3111b843-fa8c-465e-8136-457d92349d7d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:55.055658 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:55.065129 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 556/1115] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:00:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:00:55.071284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189b7c8-e522-43c0-86f3-dbce0d539b35 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:55.169204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f189b7c8-e522-43c0-86f3-dbce0d539b35 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:55.194729 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:55.203452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fade9c-6e24-4939-8e2f-14c614bdaae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:55.204080 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fade9c-6e24-4939-8e2f-14c614bdaae4 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:55.204356 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fade9c-6e24-4939-8e2f-14c614bdaae4 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:55.204628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fade9c-6e24-4939-8e2f-14c614bdaae4 None admin] PATH_INFO: `/v3/projects/51958836cb824c6ab0e3ab66cd624f34` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:55.205772 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fade9c-6e24-4939-8e2f-14c614bdaae4 None admin] RBAC: Authorizing `identity:get_project(project_id=51958836cb824c6ab0e3ab66cd624f34)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:55.207197 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fade9c-6e24-4939-8e2f-14c614bdaae4 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:55.208829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 560/1116] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 19:00:55 2022] GET /identity/v3/projects/51958836cb824c6ab0e3ab66cd624f34 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:00:55.216237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111b843-fa8c-465e-8136-457d92349d7d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:55.318134 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3111b843-fa8c-465e-8136-457d92349d7d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:55.352335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a54c87e-f74f-44ed-884b-f58c044e1bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:55.352921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54c87e-f74f-44ed-884b-f58c044e1bde None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:55.353213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54c87e-f74f-44ed-884b-f58c044e1bde None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:55.353488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a54c87e-f74f-44ed-884b-f58c044e1bde None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:55.354227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54c87e-f74f-44ed-884b-f58c044e1bde None admin] RBAC: Adding query filter params (role.id=470b660efbe04083a803e79d8b647160, scope.project.id=51958836cb824c6ab0e3ab66cd624f34, user.id=4d166a3a044642d489c3bdf5122d7613) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:00:55.354524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54c87e-f74f-44ed-884b-f58c044e1bde None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:55.357603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a54c87e-f74f-44ed-884b-f58c044e1bde None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:55.363041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 557/1117] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:00:55 2022] GET /identity/v3/role_assignments?user.id=4d166a3a044642d489c3bdf5122d7613&scope.project.id=51958836cb824c6ab0e3ab66cd624f34&role.id=470b660efbe04083a803e79d8b647160 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:57.229529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17e95a-90ea-421f-9915-98c495802ec8 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:57.230050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17e95a-90ea-421f-9915-98c495802ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:57.230050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c17e95a-90ea-421f-9915-98c495802ec8 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:57.231470 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 561/1118] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:00:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:00:57.237310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d010ae-45b6-4c5c-9d19-461e152141ba None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:57.237735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d010ae-45b6-4c5c-9d19-461e152141ba None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:57.237864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d010ae-45b6-4c5c-9d19-461e152141ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:57.255768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-50d010ae-45b6-4c5c-9d19-461e152141ba None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:00:57.256969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-50d010ae-45b6-4c5c-9d19-461e152141ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:00:57.305962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:57.322294 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 558/1119] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:00:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:00:57.593036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17e95a-90ea-421f-9915-98c495802ec8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:57.691408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c17e95a-90ea-421f-9915-98c495802ec8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:57.726130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a87ece-070c-46a8-802a-bfd015c4d244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:57.726749 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a87ece-070c-46a8-802a-bfd015c4d244 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:57.727026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a87ece-070c-46a8-802a-bfd015c4d244 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:57.727294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a87ece-070c-46a8-802a-bfd015c4d244 None admin] PATH_INFO: `/v3/roles/470b660efbe04083a803e79d8b647160` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:57.728664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a87ece-070c-46a8-802a-bfd015c4d244 None admin] RBAC: Authorizing `identity:get_role(role_id=470b660efbe04083a803e79d8b647160)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:57.730039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a87ece-070c-46a8-802a-bfd015c4d244 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:57.731409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 562/1120] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 19:00:57 2022] GET /identity/v3/roles/470b660efbe04083a803e79d8b647160 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:57.737409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d010ae-45b6-4c5c-9d19-461e152141ba None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:57.833011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d010ae-45b6-4c5c-9d19-461e152141ba None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:57.865029 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8c409-da29-42ef-8fc3-491f233e8b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:57.865620 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8c409-da29-42ef-8fc3-491f233e8b1f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:57.865898 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8c409-da29-42ef-8fc3-491f233e8b1f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:57.866170 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c8c409-da29-42ef-8fc3-491f233e8b1f None admin] PATH_INFO: `/v3/users/4d166a3a044642d489c3bdf5122d7613` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:57.867268 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8c409-da29-42ef-8fc3-491f233e8b1f None admin] RBAC: Authorizing `identity:get_user(user_id=4d166a3a044642d489c3bdf5122d7613)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:57.868684 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c8c409-da29-42ef-8fc3-491f233e8b1f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:57.870271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 559/1121] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 19:00:57 2022] GET /identity/v3/users/4d166a3a044642d489c3bdf5122d7613 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:57.876726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a87ece-070c-46a8-802a-bfd015c4d244 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:57.937133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:57.973914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a87ece-070c-46a8-802a-bfd015c4d244 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:58.006828 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a0acb-2126-4d45-b3b5-578ca670ad6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:58.007438 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a0acb-2126-4d45-b3b5-578ca670ad6d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:58.007715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a0acb-2126-4d45-b3b5-578ca670ad6d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:58.007985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a0acb-2126-4d45-b3b5-578ca670ad6d None admin] PATH_INFO: `/v3/projects/51958836cb824c6ab0e3ab66cd624f34` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:58.009074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a0acb-2126-4d45-b3b5-578ca670ad6d None admin] RBAC: Authorizing `identity:get_project(project_id=51958836cb824c6ab0e3ab66cd624f34)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:58.010525 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0a0acb-2126-4d45-b3b5-578ca670ad6d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:58.012669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 563/1122] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 19:00:57 2022] GET /identity/v3/projects/51958836cb824c6ab0e3ab66cd624f34 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:00:58.018568 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8c409-da29-42ef-8fc3-491f233e8b1f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:00:58.071872 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:00:58.114944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c8c409-da29-42ef-8fc3-491f233e8b1f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:00:58.147824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd27544-7d24-4b24-b904-446ffca23a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:00:58.148404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd27544-7d24-4b24-b904-446ffca23a87 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:00:58.148681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd27544-7d24-4b24-b904-446ffca23a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:00:58.148950 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd27544-7d24-4b24-b904-446ffca23a87 None admin] PATH_INFO: `/v3/projects/51958836cb824c6ab0e3ab66cd624f34/users/4d166a3a044642d489c3bdf5122d7613/roles/470b660efbe04083a803e79d8b647160` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:00:58.151082 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd27544-7d24-4b24-b904-446ffca23a87 None admin] RBAC: Authorizing `identity:create_grant(project_id=51958836cb824c6ab0e3ab66cd624f34, user_id=4d166a3a044642d489c3bdf5122d7613, role_id=470b660efbe04083a803e79d8b647160)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:00:58.152522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd27544-7d24-4b24-b904-446ffca23a87 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:00:58.164740 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 560/1123] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 19:00:58 2022] PUT /identity/v3/projects/51958836cb824c6ab0e3ab66cd624f34/users/4d166a3a044642d489c3bdf5122d7613/roles/470b660efbe04083a803e79d8b647160 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:01:00.044810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d416b-d457-4d02-b9e8-c5313f1d444b None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:00.045426 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d416b-d457-4d02-b9e8-c5313f1d444b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:00.045426 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122d416b-d457-4d02-b9e8-c5313f1d444b None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:00.046715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 564/1124] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:01:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:01:00.051857 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a968d9-b409-4b09-8527-02d182367d19 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:00.052298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a968d9-b409-4b09-8527-02d182367d19 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:00.052396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a968d9-b409-4b09-8527-02d182367d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:00.070169 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b3a968d9-b409-4b09-8527-02d182367d19 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:01:00.071382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b3a968d9-b409-4b09-8527-02d182367d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:01:00.135936 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 561/1125] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:01:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:01:00.406535 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d416b-d457-4d02-b9e8-c5313f1d444b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:00.503798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122d416b-d457-4d02-b9e8-c5313f1d444b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:00.532457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:00.537580 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc86ab2-7f6c-4737-87bd-b0157bc321d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:00.538189 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc86ab2-7f6c-4737-87bd-b0157bc321d3 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:00.538465 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc86ab2-7f6c-4737-87bd-b0157bc321d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:00.538734 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc86ab2-7f6c-4737-87bd-b0157bc321d3 None admin] PATH_INFO: `/v3/roles/470b660efbe04083a803e79d8b647160` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:00.540098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc86ab2-7f6c-4737-87bd-b0157bc321d3 None admin] RBAC: Authorizing `identity:get_role(role_id=470b660efbe04083a803e79d8b647160)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:00.541471 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc86ab2-7f6c-4737-87bd-b0157bc321d3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:00.542790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 565/1126] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 19:01:00 2022] GET /identity/v3/roles/470b660efbe04083a803e79d8b647160 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:00.548427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a968d9-b409-4b09-8527-02d182367d19 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:00.593466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:00.645762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a968d9-b409-4b09-8527-02d182367d19 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:00.679030 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ecdf6-fe46-4f15-90ed-3e2323cfabf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:00.679577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ecdf6-fe46-4f15-90ed-3e2323cfabf8 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:00.679851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ecdf6-fe46-4f15-90ed-3e2323cfabf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:00.680117 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692ecdf6-fe46-4f15-90ed-3e2323cfabf8 None admin] PATH_INFO: `/v3/users/4d166a3a044642d489c3bdf5122d7613` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:00.681223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ecdf6-fe46-4f15-90ed-3e2323cfabf8 None admin] RBAC: Authorizing `identity:get_user(user_id=4d166a3a044642d489c3bdf5122d7613)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:00.682627 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692ecdf6-fe46-4f15-90ed-3e2323cfabf8 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:00.684254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 562/1127] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 19:01:00 2022] GET /identity/v3/users/4d166a3a044642d489c3bdf5122d7613 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:00.690166 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc86ab2-7f6c-4737-87bd-b0157bc321d3 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:00.787112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc86ab2-7f6c-4737-87bd-b0157bc321d3 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:00.819515 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f78b4-4a01-439d-93e6-479fb0a1774a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:00.820100 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f78b4-4a01-439d-93e6-479fb0a1774a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:00.820373 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f78b4-4a01-439d-93e6-479fb0a1774a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:00.820639 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984f78b4-4a01-439d-93e6-479fb0a1774a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:00.882883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:00.885070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f78b4-4a01-439d-93e6-479fb0a1774a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:00.887479 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984f78b4-4a01-439d-93e6-479fb0a1774a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:00.984693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 566/1128] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:01:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:01:00.990576 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ecdf6-fe46-4f15-90ed-3e2323cfabf8 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:01.091004 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692ecdf6-fe46-4f15-90ed-3e2323cfabf8 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:01.103514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:01.123635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f892ed8-4e50-4d07-bfd7-81d0dfbb0fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:01.124498 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f892ed8-4e50-4d07-bfd7-81d0dfbb0fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:01.124772 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f892ed8-4e50-4d07-bfd7-81d0dfbb0fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:01.125037 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f892ed8-4e50-4d07-bfd7-81d0dfbb0fd1 None admin] PATH_INFO: `/v3/projects/51958836cb824c6ab0e3ab66cd624f34` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:01.126148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f892ed8-4e50-4d07-bfd7-81d0dfbb0fd1 None admin] RBAC: Authorizing `identity:get_project(project_id=51958836cb824c6ab0e3ab66cd624f34)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:01.127551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f892ed8-4e50-4d07-bfd7-81d0dfbb0fd1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:01.129257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 563/1129] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 19:01:00 2022] GET /identity/v3/projects/51958836cb824c6ab0e3ab66cd624f34 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:01.135465 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f78b4-4a01-439d-93e6-479fb0a1774a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:01.231591 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:01.234768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984f78b4-4a01-439d-93e6-479fb0a1774a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:01.269148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5cb40df-2db4-489f-8e39-5270e0dc13ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:01.269759 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb40df-2db4-489f-8e39-5270e0dc13ed None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:01.270029 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb40df-2db4-489f-8e39-5270e0dc13ed None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:01.270296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5cb40df-2db4-489f-8e39-5270e0dc13ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:01.271002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb40df-2db4-489f-8e39-5270e0dc13ed None admin] RBAC: Adding query filter params (role.id=470b660efbe04083a803e79d8b647160, scope.project.id=51958836cb824c6ab0e3ab66cd624f34, user.id=4d166a3a044642d489c3bdf5122d7613) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:01:01.271302 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb40df-2db4-489f-8e39-5270e0dc13ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:01.272695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5cb40df-2db4-489f-8e39-5270e0dc13ed None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:01.278846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 567/1130] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:01:01 2022] GET /identity/v3/role_assignments?user.id=4d166a3a044642d489c3bdf5122d7613&scope.project.id=51958836cb824c6ab0e3ab66cd624f34&role.id=470b660efbe04083a803e79d8b647160 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:03.195294 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a05c4e-a286-4ab7-8f8d-a91f9ac646ad None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:03.195756 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a05c4e-a286-4ab7-8f8d-a91f9ac646ad None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:03.195845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a05c4e-a286-4ab7-8f8d-a91f9ac646ad None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:03.197166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 564/1131] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:01:03.202942 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0580c7-a1da-4ef9-9744-5f364f4d1c50 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:03.203353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0580c7-a1da-4ef9-9744-5f364f4d1c50 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:03.203482 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0580c7-a1da-4ef9-9744-5f364f4d1c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:03.221508 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6f0580c7-a1da-4ef9-9744-5f364f4d1c50 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:01:03.222688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6f0580c7-a1da-4ef9-9744-5f364f4d1c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:01:03.286655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 568/1132] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:01:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:01:03.564807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a05c4e-a286-4ab7-8f8d-a91f9ac646ad None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:03.663011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a05c4e-a286-4ab7-8f8d-a91f9ac646ad None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:03.698438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd78642b-dad8-4198-a37a-5cbf20c71a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:03.699079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78642b-dad8-4198-a37a-5cbf20c71a93 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:03.699357 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78642b-dad8-4198-a37a-5cbf20c71a93 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:03.699623 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd78642b-dad8-4198-a37a-5cbf20c71a93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:03.700710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78642b-dad8-4198-a37a-5cbf20c71a93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:03.702162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd78642b-dad8-4198-a37a-5cbf20c71a93 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:03.703712 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 565/1133] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:01:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:03.709570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0580c7-a1da-4ef9-9744-5f364f4d1c50 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:03.800050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:03.807060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0580c7-a1da-4ef9-9744-5f364f4d1c50 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:03.839291 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43627b-b833-43d3-a520-3cfd36110cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:03.840304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43627b-b833-43d3-a520-3cfd36110cec None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:03.840579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43627b-b833-43d3-a520-3cfd36110cec None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:03.840845 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43627b-b833-43d3-a520-3cfd36110cec None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:03.841502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43627b-b833-43d3-a520-3cfd36110cec None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:03.842919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43627b-b833-43d3-a520-3cfd36110cec None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:03.868108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 569/1134] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 19:01:03 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:01:05.790374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9446b1f-a58a-4c03-9d75-92cfa65f1914 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:05.790911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9446b1f-a58a-4c03-9d75-92cfa65f1914 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:05.790911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9446b1f-a58a-4c03-9d75-92cfa65f1914 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:05.792240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 566/1135] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:01:05.797491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b63c0-3874-4f95-ad92-6a2f666229a8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:05.797869 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b63c0-3874-4f95-ad92-6a2f666229a8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:05.798033 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b63c0-3874-4f95-ad92-6a2f666229a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:05.815775 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-822b63c0-3874-4f95-ad92-6a2f666229a8 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:01:05.817069 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-822b63c0-3874-4f95-ad92-6a2f666229a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:01:05.881791 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 570/1136] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:01:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:01:06.146771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9446b1f-a58a-4c03-9d75-92cfa65f1914 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:06.195800 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:06.246480 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9446b1f-a58a-4c03-9d75-92cfa65f1914 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:06.279316 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf24b98f-10eb-4732-ac8b-426521044041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:06.279925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24b98f-10eb-4732-ac8b-426521044041 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:06.280201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24b98f-10eb-4732-ac8b-426521044041 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:06.280468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf24b98f-10eb-4732-ac8b-426521044041 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:06.281579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24b98f-10eb-4732-ac8b-426521044041 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:06.283000 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf24b98f-10eb-4732-ac8b-426521044041 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:06.284528 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 567/1137] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:01:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:06.290278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b63c0-3874-4f95-ad92-6a2f666229a8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:06.362968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:06.386616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b63c0-3874-4f95-ad92-6a2f666229a8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:06.419085 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f70546d-c6f1-4905-845a-93d49f3d385a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:06.419615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70546d-c6f1-4905-845a-93d49f3d385a None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:06.419889 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70546d-c6f1-4905-845a-93d49f3d385a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:06.420155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f70546d-c6f1-4905-845a-93d49f3d385a None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:06.420789 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70546d-c6f1-4905-845a-93d49f3d385a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:06.422227 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f70546d-c6f1-4905-845a-93d49f3d385a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:06.443765 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 571/1138] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 19:01:06 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:01:08.372814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9cfdb-84ce-4620-8d3e-c92015615709 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:08.373390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9cfdb-84ce-4620-8d3e-c92015615709 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:08.373390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9cfdb-84ce-4620-8d3e-c92015615709 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:08.374971 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 568/1139] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:01:08.380112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa3f10-5891-4d1e-ac88-e313c3a6a441 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:08.380539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa3f10-5891-4d1e-ac88-e313c3a6a441 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:08.380657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa3f10-5891-4d1e-ac88-e313c3a6a441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:08.398412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3daa3f10-5891-4d1e-ac88-e313c3a6a441 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:01:08.399592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3daa3f10-5891-4d1e-ac88-e313c3a6a441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:01:08.465444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 572/1140] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:01:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:01:08.732832 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9cfdb-84ce-4620-8d3e-c92015615709 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:08.832716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9cfdb-84ce-4620-8d3e-c92015615709 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:08.848750 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:08.865952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec011c-8b63-444e-a0a4-303a7f567df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:08.866560 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec011c-8b63-444e-a0a4-303a7f567df2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:08.866838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec011c-8b63-444e-a0a4-303a7f567df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:08.867105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaec011c-8b63-444e-a0a4-303a7f567df2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:08.877074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaec011c-8b63-444e-a0a4-303a7f567df2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:01:08.881795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-aaec011c-8b63-444e-a0a4-303a7f567df2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:01:08.881795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:01:08.881795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:01:08.881795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:01:08.881795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:01:08.881795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:01:08.881795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:01:08.881795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:01:08.881795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:01:08.881795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:01:08.881795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:01:08.881795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:01:08.881795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:01:08.881795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:01:08.881795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:01:08.881795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:01:08.882934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:01:08.882934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:01:08.882934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:01:08.882934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:01:08.882934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:01:08.882934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:01:08.882934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:01:08.882934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:01:08.882934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:01:08.882934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:01:08.882934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:01:08.882934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:01:08.882934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:01:08.882934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:01:08.882934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:01:08.882934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:01:08.884003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:01:08.884003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:01:08.884003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:01:08.884003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:01:08.884003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:01:08.884003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:01:08.884003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:01:08.884003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:01:08.884003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:01:08.884003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:01:08.884003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:01:08.884003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:01:08.884003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 569/1141] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:01:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:01:08.889492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa3f10-5891-4d1e-ac88-e313c3a6a441 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:08.948045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:08.987879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa3f10-5891-4d1e-ac88-e313c3a6a441 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:09.022336 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dbdfe-b2b8-40e3-9f68-5a0534060aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:09.022926 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dbdfe-b2b8-40e3-9f68-5a0534060aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:09.023204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dbdfe-b2b8-40e3-9f68-5a0534060aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:09.023475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747dbdfe-b2b8-40e3-9f68-5a0534060aa5 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:09.024122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747dbdfe-b2b8-40e3-9f68-5a0534060aa5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:01:09.024420 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747dbdfe-b2b8-40e3-9f68-5a0534060aa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:09.025806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747dbdfe-b2b8-40e3-9f68-5a0534060aa5 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:09.033666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 573/1142] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:01:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:09.039376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec011c-8b63-444e-a0a4-303a7f567df2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:09.135089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaec011c-8b63-444e-a0a4-303a7f567df2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:09.167670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82266ae2-7c02-46c6-9aa6-27f39a5d6f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:09.168272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82266ae2-7c02-46c6-9aa6-27f39a5d6f42 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:09.168551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82266ae2-7c02-46c6-9aa6-27f39a5d6f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:09.168821 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82266ae2-7c02-46c6-9aa6-27f39a5d6f42 None admin] PATH_INFO: `/v3/users/a79a9418eadc400d866b9c48860637fc` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:09.181592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82266ae2-7c02-46c6-9aa6-27f39a5d6f42 None admin] RBAC: Authorizing `identity:get_user(user_id=a79a9418eadc400d866b9c48860637fc)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:09.183010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82266ae2-7c02-46c6-9aa6-27f39a5d6f42 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:09.185074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 570/1143] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 19:01:09 2022] GET /identity/v3/users/a79a9418eadc400d866b9c48860637fc => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:09.190850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dbdfe-b2b8-40e3-9f68-5a0534060aa5 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:09.286214 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747dbdfe-b2b8-40e3-9f68-5a0534060aa5 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:09.307671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:09.318642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f79073-d889-4443-b8be-51f9b760cd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:09.319226 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f79073-d889-4443-b8be-51f9b760cd69 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:09.319497 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f79073-d889-4443-b8be-51f9b760cd69 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:09.319760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f79073-d889-4443-b8be-51f9b760cd69 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:09.383808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f79073-d889-4443-b8be-51f9b760cd69 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:09.385245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f79073-d889-4443-b8be-51f9b760cd69 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:09.477611 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 574/1144] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:01:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:01:09.483994 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82266ae2-7c02-46c6-9aa6-27f39a5d6f42 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:09.517474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:09.582114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82266ae2-7c02-46c6-9aa6-27f39a5d6f42 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:09.615402 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b3566-6d69-4057-9bbe-65eda1672105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:09.616017 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b3566-6d69-4057-9bbe-65eda1672105 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:09.616294 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b3566-6d69-4057-9bbe-65eda1672105 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:09.616566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b3566-6d69-4057-9bbe-65eda1672105 None admin] PATH_INFO: `/v3/projects/526929ddef3941f89974f69263c9b233` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:09.617673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b3566-6d69-4057-9bbe-65eda1672105 None admin] RBAC: Authorizing `identity:get_project(project_id=526929ddef3941f89974f69263c9b233)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:09.619080 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b3566-6d69-4057-9bbe-65eda1672105 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:09.620794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 571/1145] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 19:01:09 2022] GET /identity/v3/projects/526929ddef3941f89974f69263c9b233 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:09.627943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f79073-d889-4443-b8be-51f9b760cd69 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:09.651481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:09.723780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f79073-d889-4443-b8be-51f9b760cd69 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:09.756313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6c8ecd-c6a7-4ee6-91aa-2f08fc56c238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:09.756896 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c8ecd-c6a7-4ee6-91aa-2f08fc56c238 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:09.757166 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c8ecd-c6a7-4ee6-91aa-2f08fc56c238 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:09.757461 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6c8ecd-c6a7-4ee6-91aa-2f08fc56c238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:09.758164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c8ecd-c6a7-4ee6-91aa-2f08fc56c238 None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=526929ddef3941f89974f69263c9b233, user.id=a79a9418eadc400d866b9c48860637fc) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:01:09.758462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c8ecd-c6a7-4ee6-91aa-2f08fc56c238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:09.759856 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6c8ecd-c6a7-4ee6-91aa-2f08fc56c238 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:09.765044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 575/1146] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:01:09 2022] GET /identity/v3/role_assignments?user.id=a79a9418eadc400d866b9c48860637fc&scope.project.id=526929ddef3941f89974f69263c9b233&role.id=3580c894dd974d35bc95848b8ba22598 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:11.617473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc54be-b927-4283-a5ff-68ea374c46ef None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:11.618028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc54be-b927-4283-a5ff-68ea374c46ef None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:11.618028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dc54be-b927-4283-a5ff-68ea374c46ef None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:11.619338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 572/1147] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:01:11.624760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafccd61-00a0-4a5b-b13b-47f802cf0fe4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:11.625164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafccd61-00a0-4a5b-b13b-47f802cf0fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:11.625325 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafccd61-00a0-4a5b-b13b-47f802cf0fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:11.643366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-cafccd61-00a0-4a5b-b13b-47f802cf0fe4 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:01:11.644537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cafccd61-00a0-4a5b-b13b-47f802cf0fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:01:11.708919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 576/1148] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:01:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:01:11.973140 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc54be-b927-4283-a5ff-68ea374c46ef None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:12.070539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dc54be-b927-4283-a5ff-68ea374c46ef None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:12.071997 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:12.103569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f81a8-c3d3-48e3-a305-b634badafb5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:12.104179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f81a8-c3d3-48e3-a305-b634badafb5b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:12.104455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f81a8-c3d3-48e3-a305-b634badafb5b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:12.104725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067f81a8-c3d3-48e3-a305-b634badafb5b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:12.114324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067f81a8-c3d3-48e3-a305-b634badafb5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:01:12.118983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-067f81a8-c3d3-48e3-a305-b634badafb5b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:01:12.118983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:01:12.118983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:01:12.118983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:01:12.118983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:01:12.118983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:01:12.118983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:01:12.118983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:01:12.118983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:01:12.118983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:01:12.118983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:01:12.118983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:01:12.118983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:01:12.118983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:01:12.118983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:01:12.118983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:01:12.120156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:01:12.120156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:01:12.120156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:01:12.120156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:01:12.120156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:01:12.120156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:01:12.120156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:01:12.120156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:01:12.120156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:01:12.120156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:01:12.120156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:01:12.120156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:01:12.120156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:01:12.120156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:01:12.120156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:01:12.120156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:01:12.121609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:01:12.121609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:01:12.121609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:01:12.121609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:01:12.121609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:01:12.121609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:01:12.121609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:01:12.121609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:01:12.121609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:01:12.121609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:01:12.121609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:01:12.121609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:01:12.121609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 573/1149] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:01:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:01:12.126693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafccd61-00a0-4a5b-b13b-47f802cf0fe4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:12.145899 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:12.225449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafccd61-00a0-4a5b-b13b-47f802cf0fe4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:12.258311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1977a3-25cc-4a00-8813-792b4fbbbd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:12.258905 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1977a3-25cc-4a00-8813-792b4fbbbd45 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:12.259182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1977a3-25cc-4a00-8813-792b4fbbbd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:12.259452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1977a3-25cc-4a00-8813-792b4fbbbd45 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:12.260100 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1977a3-25cc-4a00-8813-792b4fbbbd45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:01:12.260398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1977a3-25cc-4a00-8813-792b4fbbbd45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:12.262471 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1977a3-25cc-4a00-8813-792b4fbbbd45 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:12.270566 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 577/1150] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:01:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:12.276296 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f81a8-c3d3-48e3-a305-b634badafb5b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:12.377172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067f81a8-c3d3-48e3-a305-b634badafb5b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:12.409074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac0952-6613-4e25-9794-cda6bdf74a90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:12.409811 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac0952-6613-4e25-9794-cda6bdf74a90 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:12.410087 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac0952-6613-4e25-9794-cda6bdf74a90 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:12.410354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ac0952-6613-4e25-9794-cda6bdf74a90 None admin] PATH_INFO: `/v3/users/a79a9418eadc400d866b9c48860637fc` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:12.411445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac0952-6613-4e25-9794-cda6bdf74a90 None admin] RBAC: Authorizing `identity:get_user(user_id=a79a9418eadc400d866b9c48860637fc)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:12.412845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ac0952-6613-4e25-9794-cda6bdf74a90 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:12.414411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 574/1151] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 19:01:12 2022] GET /identity/v3/users/a79a9418eadc400d866b9c48860637fc => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:12.420768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1977a3-25cc-4a00-8813-792b4fbbbd45 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:12.502120 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:12.521178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1977a3-25cc-4a00-8813-792b4fbbbd45 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:12.553364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff27905-d108-4f60-910e-258f748bf3d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:12.553976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff27905-d108-4f60-910e-258f748bf3d1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:12.554250 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff27905-d108-4f60-910e-258f748bf3d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:12.554518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff27905-d108-4f60-910e-258f748bf3d1 None admin] PATH_INFO: `/v3/projects/526929ddef3941f89974f69263c9b233` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:12.555608 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff27905-d108-4f60-910e-258f748bf3d1 None admin] RBAC: Authorizing `identity:get_project(project_id=526929ddef3941f89974f69263c9b233)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:12.557021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff27905-d108-4f60-910e-258f748bf3d1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:12.559247 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 578/1152] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 19:01:12 2022] GET /identity/v3/projects/526929ddef3941f89974f69263c9b233 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:12.565084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac0952-6613-4e25-9794-cda6bdf74a90 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:12.593913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:12.661564 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ac0952-6613-4e25-9794-cda6bdf74a90 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:12.694329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d6a31-33ff-4239-8089-f916f64a07d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:12.694959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d6a31-33ff-4239-8089-f916f64a07d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:12.695231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d6a31-33ff-4239-8089-f916f64a07d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:12.695496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d6a31-33ff-4239-8089-f916f64a07d9 None admin] PATH_INFO: `/v3/projects/526929ddef3941f89974f69263c9b233/users/a79a9418eadc400d866b9c48860637fc/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:12.697356 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d6a31-33ff-4239-8089-f916f64a07d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=526929ddef3941f89974f69263c9b233, user_id=a79a9418eadc400d866b9c48860637fc, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:12.698793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d6a31-33ff-4239-8089-f916f64a07d9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:12.710725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 575/1153] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 19:01:12 2022] PUT /identity/v3/projects/526929ddef3941f89974f69263c9b233/users/a79a9418eadc400d866b9c48860637fc/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:01:14.576009 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cec94-5a29-4b7c-a241-1d5b371a29ba None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:14.576565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cec94-5a29-4b7c-a241-1d5b371a29ba None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:14.576565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cec94-5a29-4b7c-a241-1d5b371a29ba None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:14.577928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 579/1154] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:01:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:01:14.582981 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa469802-764c-4b95-ac21-9715bd13bd19 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:14.583384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa469802-764c-4b95-ac21-9715bd13bd19 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:14.583516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa469802-764c-4b95-ac21-9715bd13bd19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:14.601764 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-aa469802-764c-4b95-ac21-9715bd13bd19 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:01:14.602923 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-aa469802-764c-4b95-ac21-9715bd13bd19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:01:14.667986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 576/1155] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:01:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:01:14.939459 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cec94-5a29-4b7c-a241-1d5b371a29ba None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:15.038480 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cec94-5a29-4b7c-a241-1d5b371a29ba None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:15.070949 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3de060-2c88-4304-b97d-adf910bb124f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:15.071557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3de060-2c88-4304-b97d-adf910bb124f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:15.071834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3de060-2c88-4304-b97d-adf910bb124f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:15.072103 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3de060-2c88-4304-b97d-adf910bb124f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:15.082412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3de060-2c88-4304-b97d-adf910bb124f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:01:15.087131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-bf3de060-2c88-4304-b97d-adf910bb124f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:01:15.087131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:01:15.087131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:01:15.087131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:01:15.087131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:01:15.087131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:01:15.087131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:01:15.087131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:01:15.087131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:01:15.087131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:01:15.087131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:01:15.087131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:01:15.087131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:01:15.087131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:01:15.087131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:01:15.087131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:01:15.088219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:01:15.088219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:01:15.088219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:01:15.088219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:01:15.088219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:01:15.088219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:01:15.088219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:01:15.088219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:01:15.088219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:01:15.088219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:01:15.088219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:01:15.088219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:01:15.088219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:01:15.088219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:01:15.088219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:01:15.088219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:01:15.089282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:01:15.089282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:01:15.089282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:01:15.089282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:01:15.089282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:01:15.089282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:01:15.089282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:01:15.089282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:01:15.089282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:01:15.089282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:01:15.089282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:01:15.089282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:01:15.089282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 580/1156] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:01:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:01:15.094353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa469802-764c-4b95-ac21-9715bd13bd19 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:15.114507 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:15.189554 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa469802-764c-4b95-ac21-9715bd13bd19 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:15.222149 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c58d87-d797-4e90-9f61-4ced28fb6d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:15.222739 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c58d87-d797-4e90-9f61-4ced28fb6d5d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:15.223012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c58d87-d797-4e90-9f61-4ced28fb6d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:15.223280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c58d87-d797-4e90-9f61-4ced28fb6d5d None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:15.223930 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c58d87-d797-4e90-9f61-4ced28fb6d5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:01:15.224224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c58d87-d797-4e90-9f61-4ced28fb6d5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:15.225596 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c58d87-d797-4e90-9f61-4ced28fb6d5d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:15.233338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 577/1157] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:01:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:15.239160 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3de060-2c88-4304-b97d-adf910bb124f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:15.251021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:15.338151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3de060-2c88-4304-b97d-adf910bb124f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:15.371243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135563e-23b5-4507-b8a8-2c1512dbe5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:15.371862 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135563e-23b5-4507-b8a8-2c1512dbe5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:15.372140 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135563e-23b5-4507-b8a8-2c1512dbe5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:15.372413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135563e-23b5-4507-b8a8-2c1512dbe5e1 None admin] PATH_INFO: `/v3/users/a79a9418eadc400d866b9c48860637fc` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:15.373555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135563e-23b5-4507-b8a8-2c1512dbe5e1 None admin] RBAC: Authorizing `identity:get_user(user_id=a79a9418eadc400d866b9c48860637fc)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:15.375224 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3135563e-23b5-4507-b8a8-2c1512dbe5e1 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:15.376782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 581/1158] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 19:01:15 2022] GET /identity/v3/users/a79a9418eadc400d866b9c48860637fc => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:15.382651 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c58d87-d797-4e90-9f61-4ced28fb6d5d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:15.460746 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:15.478558 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c58d87-d797-4e90-9f61-4ced28fb6d5d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:15.513673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073f9b2-059b-42f4-9c60-d2ec87efa912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:15.514259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073f9b2-059b-42f4-9c60-d2ec87efa912 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:15.514533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073f9b2-059b-42f4-9c60-d2ec87efa912 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:15.514801 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f073f9b2-059b-42f4-9c60-d2ec87efa912 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:15.580596 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073f9b2-059b-42f4-9c60-d2ec87efa912 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:15.582071 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f073f9b2-059b-42f4-9c60-d2ec87efa912 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:15.659517 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:15.678269 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 578/1159] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:01:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 31 19:01:15.684753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135563e-23b5-4507-b8a8-2c1512dbe5e1 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:15.759219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:15.782087 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135563e-23b5-4507-b8a8-2c1512dbe5e1 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:15.814636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ad6ae-6426-42bf-8dbe-08fb4b2f0707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:15.815249 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ad6ae-6426-42bf-8dbe-08fb4b2f0707 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:15.815530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ad6ae-6426-42bf-8dbe-08fb4b2f0707 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:15.815801 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ad6ae-6426-42bf-8dbe-08fb4b2f0707 None admin] PATH_INFO: `/v3/projects/526929ddef3941f89974f69263c9b233` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:15.816890 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ad6ae-6426-42bf-8dbe-08fb4b2f0707 None admin] RBAC: Authorizing `identity:get_project(project_id=526929ddef3941f89974f69263c9b233)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:15.818333 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393ad6ae-6426-42bf-8dbe-08fb4b2f0707 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:15.819938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 582/1160] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 19:01:15 2022] GET /identity/v3/projects/526929ddef3941f89974f69263c9b233 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:15.826025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073f9b2-059b-42f4-9c60-d2ec87efa912 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:15.920776 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f073f9b2-059b-42f4-9c60-d2ec87efa912 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:15.952638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c3c63b-eaaf-4aa9-8832-1b122d953a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:15.953231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3c63b-eaaf-4aa9-8832-1b122d953a07 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:15.953510 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3c63b-eaaf-4aa9-8832-1b122d953a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:15.953776 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c3c63b-eaaf-4aa9-8832-1b122d953a07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:15.954477 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3c63b-eaaf-4aa9-8832-1b122d953a07 None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=526929ddef3941f89974f69263c9b233, user.id=a79a9418eadc400d866b9c48860637fc) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:01:15.954777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3c63b-eaaf-4aa9-8832-1b122d953a07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:15.956180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c3c63b-eaaf-4aa9-8832-1b122d953a07 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:15.961624 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 579/1161] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:01:15 2022] GET /identity/v3/role_assignments?user.id=a79a9418eadc400d866b9c48860637fc&scope.project.id=526929ddef3941f89974f69263c9b233&role.id=3580c894dd974d35bc95848b8ba22598 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:17.865471 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65939c8c-9567-4e2c-a0f5-90587a976509 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:17.866036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65939c8c-9567-4e2c-a0f5-90587a976509 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:17.866036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65939c8c-9567-4e2c-a0f5-90587a976509 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:17.867331 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 583/1162] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:01:17.872479 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088ffb1-b0c7-47d4-ac96-e67e2525eb2a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:17.872910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088ffb1-b0c7-47d4-ac96-e67e2525eb2a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:17.873025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088ffb1-b0c7-47d4-ac96-e67e2525eb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:17.891387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-1088ffb1-b0c7-47d4-ac96-e67e2525eb2a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:01:17.892557 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1088ffb1-b0c7-47d4-ac96-e67e2525eb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:01:17.933343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:17.958079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 580/1163] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:01:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:01:18.220876 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65939c8c-9567-4e2c-a0f5-90587a976509 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:18.319831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65939c8c-9567-4e2c-a0f5-90587a976509 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:18.352704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44023f64-a99b-458b-8b1b-bbbd681c0e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:18.353316 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44023f64-a99b-458b-8b1b-bbbd681c0e07 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:18.353597 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44023f64-a99b-458b-8b1b-bbbd681c0e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:18.353866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44023f64-a99b-458b-8b1b-bbbd681c0e07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:18.368662 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:18.419655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44023f64-a99b-458b-8b1b-bbbd681c0e07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:18.421063 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44023f64-a99b-458b-8b1b-bbbd681c0e07 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:18.515067 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 584/1164] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:01:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:01:18.520956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088ffb1-b0c7-47d4-ac96-e67e2525eb2a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:18.615162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088ffb1-b0c7-47d4-ac96-e67e2525eb2a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:18.647785 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786c6a1-0bf0-4823-89d3-4a5e3e7a423f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:18.648387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786c6a1-0bf0-4823-89d3-4a5e3e7a423f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:18.648662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786c6a1-0bf0-4823-89d3-4a5e3e7a423f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:18.648925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786c6a1-0bf0-4823-89d3-4a5e3e7a423f None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:18.655335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786c6a1-0bf0-4823-89d3-4a5e3e7a423f None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:18.656745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f786c6a1-0bf0-4823-89d3-4a5e3e7a423f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-f786c6a1-0bf0-4823-89d3-4a5e3e7a423f None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:01:18.668181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:01:18.669400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:01:18.669400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:01:18.669400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:01:18.669400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:01:18.669400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:01:18.669400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:01:18.669400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:01:18.669400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:01:18.669400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:01:18.669400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:01:18.669400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:01:18.669400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:01:18.669400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:01:18.669400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:01:18.669400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:01:18.669400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:01:18.670286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:01:18.670286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:01:18.670286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:01:18.670286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:01:18.670286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:01:18.670286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:01:18.670286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:01:18.670286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:01:18.670286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:01:18.670286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:01:18.670286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 31 19:01:18.670286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:01:18.670286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 31 19:01:18.670286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:01:18.670286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 581/1165] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 19:01:18 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:01:18.675001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44023f64-a99b-458b-8b1b-bbbd681c0e07 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:18.719504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:18.770885 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44023f64-a99b-458b-8b1b-bbbd681c0e07 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:18.803347 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2dfa01-4c56-4893-b3b8-5cd9057087ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:18.803921 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dfa01-4c56-4893-b3b8-5cd9057087ea None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:18.804190 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dfa01-4c56-4893-b3b8-5cd9057087ea None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:18.804452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2dfa01-4c56-4893-b3b8-5cd9057087ea None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:18.805068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2dfa01-4c56-4893-b3b8-5cd9057087ea None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:01:18.805405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2dfa01-4c56-4893-b3b8-5cd9057087ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:18.806736 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2dfa01-4c56-4893-b3b8-5cd9057087ea None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:18.813533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 585/1166] 10.222.0.65 () {56 vars in 1115 bytes} [Wed Aug 31 19:01:18 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:20.684142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd556f11-b5a1-4c3c-beba-80ceae6536cc None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:20.684715 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd556f11-b5a1-4c3c-beba-80ceae6536cc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:20.684715 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd556f11-b5a1-4c3c-beba-80ceae6536cc None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:20.686030 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 582/1167] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:01:20.691234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc466-e579-4d60-9bc6-70ae7cf9a94c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:20.691645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc466-e579-4d60-9bc6-70ae7cf9a94c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:20.691756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fcc466-e579-4d60-9bc6-70ae7cf9a94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:20.710753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c0fcc466-e579-4d60-9bc6-70ae7cf9a94c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:01:20.711923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c0fcc466-e579-4d60-9bc6-70ae7cf9a94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:01:20.775623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 586/1168] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:01:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:01:21.072184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd556f11-b5a1-4c3c-beba-80ceae6536cc None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:21.107823 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:21.184467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd556f11-b5a1-4c3c-beba-80ceae6536cc None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:21.218010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c75f5e0-07ba-43d1-b7e8-de9121e460af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:21.218607 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75f5e0-07ba-43d1-b7e8-de9121e460af None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:21.218884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75f5e0-07ba-43d1-b7e8-de9121e460af None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:21.219152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c75f5e0-07ba-43d1-b7e8-de9121e460af None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:21.219797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75f5e0-07ba-43d1-b7e8-de9121e460af None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:21.221142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c75f5e0-07ba-43d1-b7e8-de9121e460af None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:21.236206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 583/1169] 10.222.0.65 () {60 vars in 1138 bytes} [Wed Aug 31 19:01:21 2022] POST /identity/v3/domains => generated 267 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:01:23.134128 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4692a6-6214-4e9a-9487-40428317dfbd None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:23.134663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4692a6-6214-4e9a-9487-40428317dfbd None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:23.134663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4692a6-6214-4e9a-9487-40428317dfbd None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:23.136002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 587/1170] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:01:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:01:23.141165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e0c0a-dd00-4dee-8bcc-d8ff9c88a115 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:23.141642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e0c0a-dd00-4dee-8bcc-d8ff9c88a115 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:23.141736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e0c0a-dd00-4dee-8bcc-d8ff9c88a115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:23.160193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-6f6e0c0a-dd00-4dee-8bcc-d8ff9c88a115 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:01:23.161383 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6f6e0c0a-dd00-4dee-8bcc-d8ff9c88a115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:01:23.226340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 584/1171] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:01:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:01:23.491141 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4692a6-6214-4e9a-9487-40428317dfbd None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:23.532567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:23.591622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4692a6-6214-4e9a-9487-40428317dfbd None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:23.624780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e14114-fb23-4a79-8e9d-306b99a61904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:23.625413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e14114-fb23-4a79-8e9d-306b99a61904 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:23.625687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e14114-fb23-4a79-8e9d-306b99a61904 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:23.625952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e14114-fb23-4a79-8e9d-306b99a61904 None admin] PATH_INFO: `/v3/domains/d79cffdd5f6543478eb4df97c0f0143f` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:23.635820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e14114-fb23-4a79-8e9d-306b99a61904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=d79cffdd5f6543478eb4df97c0f0143f)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:23.637839 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e14114-fb23-4a79-8e9d-306b99a61904 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:23.639385 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 588/1172] 10.222.0.65 () {56 vars in 1183 bytes} [Wed Aug 31 19:01:23 2022] GET /identity/v3/domains/d79cffdd5f6543478eb4df97c0f0143f => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:23.645272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e0c0a-dd00-4dee-8bcc-d8ff9c88a115 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:23.671294 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:23.743316 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e0c0a-dd00-4dee-8bcc-d8ff9c88a115 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:23.775640 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba99f9-6882-4e64-8955-a13390d9e365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:23.776223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba99f9-6882-4e64-8955-a13390d9e365 None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:23.776495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba99f9-6882-4e64-8955-a13390d9e365 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:23.777483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba99f9-6882-4e64-8955-a13390d9e365 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:23.778162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba99f9-6882-4e64-8955-a13390d9e365 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:23.779567 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba99f9-6882-4e64-8955-a13390d9e365 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:23.804926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 585/1173] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 19:01:23 2022] POST /identity/v3/projects => generated 361 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:01:25.688371 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bc7b2-a5c1-43ba-b803-3455587ef32f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:25.688908 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bc7b2-a5c1-43ba-b803-3455587ef32f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:25.688908 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bc7b2-a5c1-43ba-b803-3455587ef32f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:25.690697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 589/1174] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:01:25.695904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56f1b1-8811-4b86-b37a-f99712a7de8d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:25.696346 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56f1b1-8811-4b86-b37a-f99712a7de8d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:25.696438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d56f1b1-8811-4b86-b37a-f99712a7de8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:25.714943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9d56f1b1-8811-4b86-b37a-f99712a7de8d None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:01:25.716116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9d56f1b1-8811-4b86-b37a-f99712a7de8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:01:25.780165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 586/1175] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:01:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:01:26.050239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339bc7b2-a5c1-43ba-b803-3455587ef32f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:26.145680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:26.150614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339bc7b2-a5c1-43ba-b803-3455587ef32f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:26.184363 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cbae81-1021-4ac2-9523-1657648daebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:26.184980 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbae81-1021-4ac2-9523-1657648daebd None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:26.185287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbae81-1021-4ac2-9523-1657648daebd None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:26.185561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cbae81-1021-4ac2-9523-1657648daebd None admin] PATH_INFO: `/v3/domains/d79cffdd5f6543478eb4df97c0f0143f` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:26.186671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cbae81-1021-4ac2-9523-1657648daebd None admin] RBAC: Authorizing `identity:get_domain(domain_id=d79cffdd5f6543478eb4df97c0f0143f)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:26.188100 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cbae81-1021-4ac2-9523-1657648daebd None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:26.189697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 590/1176] 10.222.0.65 () {56 vars in 1183 bytes} [Wed Aug 31 19:01:26 2022] GET /identity/v3/domains/d79cffdd5f6543478eb4df97c0f0143f => generated 267 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:26.195452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56f1b1-8811-4b86-b37a-f99712a7de8d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:26.205369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:26.293948 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d56f1b1-8811-4b86-b37a-f99712a7de8d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:26.325985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2720ff27-3254-4999-a86a-665b0f0bdc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:26.326561 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720ff27-3254-4999-a86a-665b0f0bdc3b None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:26.326844 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720ff27-3254-4999-a86a-665b0f0bdc3b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:26.327115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2720ff27-3254-4999-a86a-665b0f0bdc3b None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:26.327749 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2720ff27-3254-4999-a86a-665b0f0bdc3b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:26.329152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2720ff27-3254-4999-a86a-665b0f0bdc3b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:26.350213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 587/1177] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 19:01:26 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:01:28.280395 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7b340-9466-4cc5-882c-0a45deb6d7c9 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:28.280989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7b340-9466-4cc5-882c-0a45deb6d7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:28.280989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b7b340-9466-4cc5-882c-0a45deb6d7c9 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:28.282415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 591/1178] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:01:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:01:28.287469 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b70213-8343-4fd1-8c9c-8d2d0470ba41 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:28.287869 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b70213-8343-4fd1-8c9c-8d2d0470ba41 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:28.288021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b70213-8343-4fd1-8c9c-8d2d0470ba41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:28.306179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-12b70213-8343-4fd1-8c9c-8d2d0470ba41 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:01:28.307361 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-12b70213-8343-4fd1-8c9c-8d2d0470ba41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:01:28.363773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:28.373683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 588/1179] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:01:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:01:28.641648 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7b340-9466-4cc5-882c-0a45deb6d7c9 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:28.741280 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b7b340-9466-4cc5-882c-0a45deb6d7c9 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:28.774570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be0145-66db-4bb8-9846-17bf2d41262c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:28.775170 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be0145-66db-4bb8-9846-17bf2d41262c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:28.775445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be0145-66db-4bb8-9846-17bf2d41262c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:28.775712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be0145-66db-4bb8-9846-17bf2d41262c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:28.785974 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34be0145-66db-4bb8-9846-17bf2d41262c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:01:28.790640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-34be0145-66db-4bb8-9846-17bf2d41262c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:01:28.790640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:01:28.790640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:01:28.790640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:01:28.790640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:01:28.790640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:01:28.790640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:01:28.790640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:01:28.790640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:01:28.790640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:01:28.790640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:01:28.790640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:01:28.790640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:01:28.790640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:01:28.790640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:01:28.790640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:01:28.791798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:01:28.791798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:01:28.791798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:01:28.791798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:01:28.791798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:01:28.791798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:01:28.791798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:01:28.791798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:01:28.791798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:01:28.791798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:01:28.791798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:01:28.791798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:01:28.791798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:01:28.791798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:01:28.791798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:01:28.791798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:01:28.793653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:01:28.793653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:01:28.793653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:01:28.793653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:01:28.793653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:01:28.793653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:01:28.793653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:01:28.793653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:01:28.793653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:01:28.793653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:01:28.793653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:01:28.793653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:01:28.793653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 592/1180] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:01:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:01:28.798154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b70213-8343-4fd1-8c9c-8d2d0470ba41 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:28.894044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b70213-8343-4fd1-8c9c-8d2d0470ba41 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:28.926461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd96d82-9118-48d2-9a8d-ade9a464e045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:28.927046 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd96d82-9118-48d2-9a8d-ade9a464e045 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:28.927317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd96d82-9118-48d2-9a8d-ade9a464e045 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:28.927579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd96d82-9118-48d2-9a8d-ade9a464e045 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:28.928223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd96d82-9118-48d2-9a8d-ade9a464e045 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:01:28.928513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd96d82-9118-48d2-9a8d-ade9a464e045 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:28.929923 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd96d82-9118-48d2-9a8d-ade9a464e045 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:28.937508 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 589/1181] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:01:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:28.943070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be0145-66db-4bb8-9846-17bf2d41262c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:28.971358 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:29.042136 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be0145-66db-4bb8-9846-17bf2d41262c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:29.074797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c47ce-43c5-42b6-bc59-99b4d8ef86d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:29.075409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c47ce-43c5-42b6-bc59-99b4d8ef86d3 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:29.075686 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c47ce-43c5-42b6-bc59-99b4d8ef86d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:29.075956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c47ce-43c5-42b6-bc59-99b4d8ef86d3 None admin] PATH_INFO: `/v3/users/22aea11508614a0c8a681be2566fe898` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:29.088403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c47ce-43c5-42b6-bc59-99b4d8ef86d3 None admin] RBAC: Authorizing `identity:get_user(user_id=22aea11508614a0c8a681be2566fe898)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:29.090478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c47ce-43c5-42b6-bc59-99b4d8ef86d3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:29.092052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 593/1182] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 19:01:28 2022] GET /identity/v3/users/22aea11508614a0c8a681be2566fe898 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:29.097944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd96d82-9118-48d2-9a8d-ade9a464e045 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:29.153378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:29.195033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd96d82-9118-48d2-9a8d-ade9a464e045 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:29.228231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2996abb-465c-49ad-b8bf-e1c42578d1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:29.228827 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2996abb-465c-49ad-b8bf-e1c42578d1de None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:29.229104 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2996abb-465c-49ad-b8bf-e1c42578d1de None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:29.229408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2996abb-465c-49ad-b8bf-e1c42578d1de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:29.294320 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2996abb-465c-49ad-b8bf-e1c42578d1de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:29.295747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2996abb-465c-49ad-b8bf-e1c42578d1de None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:29.347177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:29.390702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 590/1183] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:01:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:01:29.396606 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c47ce-43c5-42b6-bc59-99b4d8ef86d3 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:29.477411 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:29.492951 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c47ce-43c5-42b6-bc59-99b4d8ef86d3 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:29.526763 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a770b38a-9dbf-413a-9d5b-66a305acd783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:29.527548 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770b38a-9dbf-413a-9d5b-66a305acd783 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:29.527824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770b38a-9dbf-413a-9d5b-66a305acd783 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:29.528091 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a770b38a-9dbf-413a-9d5b-66a305acd783 None admin] PATH_INFO: `/v3/projects/3d7ffaaf899d4d1ba2bf4117c014b39d` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:29.529179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770b38a-9dbf-413a-9d5b-66a305acd783 None admin] RBAC: Authorizing `identity:get_project(project_id=3d7ffaaf899d4d1ba2bf4117c014b39d)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:29.530614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a770b38a-9dbf-413a-9d5b-66a305acd783 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:29.532225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 594/1184] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 19:01:29 2022] GET /identity/v3/projects/3d7ffaaf899d4d1ba2bf4117c014b39d => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:29.538476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2996abb-465c-49ad-b8bf-e1c42578d1de None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:29.640101 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2996abb-465c-49ad-b8bf-e1c42578d1de None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:29.673292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1f512-d841-4ad5-9edf-ffeca1b04e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:29.673885 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1f512-d841-4ad5-9edf-ffeca1b04e77 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:29.674160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1f512-d841-4ad5-9edf-ffeca1b04e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:29.674427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1f512-d841-4ad5-9edf-ffeca1b04e77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:29.675213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1f512-d841-4ad5-9edf-ffeca1b04e77 None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=3d7ffaaf899d4d1ba2bf4117c014b39d, user.id=22aea11508614a0c8a681be2566fe898) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:01:29.675515 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1f512-d841-4ad5-9edf-ffeca1b04e77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:29.676911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1f512-d841-4ad5-9edf-ffeca1b04e77 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:29.682192 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 591/1185] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:01:29 2022] GET /identity/v3/role_assignments?user.id=22aea11508614a0c8a681be2566fe898&scope.project.id=3d7ffaaf899d4d1ba2bf4117c014b39d&role.id=3580c894dd974d35bc95848b8ba22598 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:31.537665 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d81af-93e9-403f-816b-cb71065946ba None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:31.538223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d81af-93e9-403f-816b-cb71065946ba None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:31.538223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787d81af-93e9-403f-816b-cb71065946ba None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:31.539582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 595/1186] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:01:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:01:31.544634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af004fb4-4561-4a11-ba76-58e4f8dd8764 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:31.545104 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af004fb4-4561-4a11-ba76-58e4f8dd8764 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:31.545256 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af004fb4-4561-4a11-ba76-58e4f8dd8764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:31.563491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-af004fb4-4561-4a11-ba76-58e4f8dd8764 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:01:31.564663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-af004fb4-4561-4a11-ba76-58e4f8dd8764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:01:31.579159 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:31.630227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 592/1187] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:01:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:01:31.894561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d81af-93e9-403f-816b-cb71065946ba None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:31.992251 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787d81af-93e9-403f-816b-cb71065946ba None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:32.025129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade675ab-8a28-4098-b807-2aa6dd58979e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:32.025764 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade675ab-8a28-4098-b807-2aa6dd58979e None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:32.026041 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade675ab-8a28-4098-b807-2aa6dd58979e None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:32.026311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade675ab-8a28-4098-b807-2aa6dd58979e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:32.036044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade675ab-8a28-4098-b807-2aa6dd58979e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:01:32.040766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-ade675ab-8a28-4098-b807-2aa6dd58979e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:01:32.040766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:01:32.040766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:01:32.040766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:01:32.040766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:01:32.040766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:01:32.040766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:01:32.040766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:01:32.040766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:01:32.040766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:01:32.040766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:01:32.040766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:01:32.040766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:01:32.040766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:01:32.040766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:01:32.040766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:01:32.041886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:01:32.041886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:01:32.041886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:01:32.041886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:01:32.041886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:01:32.041886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:01:32.041886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:01:32.041886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:01:32.041886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:01:32.041886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:01:32.041886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:01:32.041886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:01:32.041886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:01:32.041886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:01:32.041886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:01:32.041886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:01:32.042797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:01:32.042797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:01:32.042797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:01:32.042797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:01:32.042797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:01:32.042797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:01:32.042797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:01:32.042797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:01:32.042797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:01:32.042797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:01:32.042797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:01:32.042797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:01:32.042797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 596/1188] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:01:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:01:32.048262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af004fb4-4561-4a11-ba76-58e4f8dd8764 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:32.142836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af004fb4-4561-4a11-ba76-58e4f8dd8764 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:32.175291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b7c2c-43d9-439f-b766-147e2b3b4d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:32.175824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b7c2c-43d9-439f-b766-147e2b3b4d3d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:32.176100 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b7c2c-43d9-439f-b766-147e2b3b4d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:32.176366 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5b7c2c-43d9-439f-b766-147e2b3b4d3d None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:32.177013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b7c2c-43d9-439f-b766-147e2b3b4d3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:01:32.177338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b7c2c-43d9-439f-b766-147e2b3b4d3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:32.178681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5b7c2c-43d9-439f-b766-147e2b3b4d3d None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:32.187081 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 593/1189] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:01:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:32.193062 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade675ab-8a28-4098-b807-2aa6dd58979e None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:32.208481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:32.292507 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade675ab-8a28-4098-b807-2aa6dd58979e None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:32.325602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48078dc-89c9-4d91-832e-6e855e81b328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:32.326208 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48078dc-89c9-4d91-832e-6e855e81b328 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:32.326482 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48078dc-89c9-4d91-832e-6e855e81b328 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:32.326748 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48078dc-89c9-4d91-832e-6e855e81b328 None admin] PATH_INFO: `/v3/users/22aea11508614a0c8a681be2566fe898` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:32.327841 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48078dc-89c9-4d91-832e-6e855e81b328 None admin] RBAC: Authorizing `identity:get_user(user_id=22aea11508614a0c8a681be2566fe898)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:32.329278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48078dc-89c9-4d91-832e-6e855e81b328 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:32.330835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 597/1190] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 19:01:32 2022] GET /identity/v3/users/22aea11508614a0c8a681be2566fe898 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:32.336611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b7c2c-43d9-439f-b766-147e2b3b4d3d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:32.351804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:32.432958 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5b7c2c-43d9-439f-b766-147e2b3b4d3d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:32.463940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bcf91a-eb78-4f13-b9a5-84cd0f2b3a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:32.464540 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcf91a-eb78-4f13-b9a5-84cd0f2b3a75 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:32.464812 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcf91a-eb78-4f13-b9a5-84cd0f2b3a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:32.465080 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bcf91a-eb78-4f13-b9a5-84cd0f2b3a75 None admin] PATH_INFO: `/v3/projects/3d7ffaaf899d4d1ba2bf4117c014b39d` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:32.466687 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcf91a-eb78-4f13-b9a5-84cd0f2b3a75 None admin] RBAC: Authorizing `identity:get_project(project_id=3d7ffaaf899d4d1ba2bf4117c014b39d)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:32.468092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bcf91a-eb78-4f13-b9a5-84cd0f2b3a75 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:32.469702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 594/1191] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 19:01:32 2022] GET /identity/v3/projects/3d7ffaaf899d4d1ba2bf4117c014b39d => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:32.475455 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48078dc-89c9-4d91-832e-6e855e81b328 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:32.552760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:32.571182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48078dc-89c9-4d91-832e-6e855e81b328 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:32.604594 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ad1b1-c683-4c96-bada-7f72cc17cf47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:32.605255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ad1b1-c683-4c96-bada-7f72cc17cf47 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:32.605530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ad1b1-c683-4c96-bada-7f72cc17cf47 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:32.605823 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ad1b1-c683-4c96-bada-7f72cc17cf47 None admin] PATH_INFO: `/v3/projects/3d7ffaaf899d4d1ba2bf4117c014b39d/users/22aea11508614a0c8a681be2566fe898/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:32.607673 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ad1b1-c683-4c96-bada-7f72cc17cf47 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d7ffaaf899d4d1ba2bf4117c014b39d, user_id=22aea11508614a0c8a681be2566fe898, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:32.609075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ad1b1-c683-4c96-bada-7f72cc17cf47 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:32.621038 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 598/1192] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 19:01:32 2022] PUT /identity/v3/projects/3d7ffaaf899d4d1ba2bf4117c014b39d/users/22aea11508614a0c8a681be2566fe898/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:01:34.471727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d791a32-ab2b-47e6-85fa-b58f88311938 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:34.472272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d791a32-ab2b-47e6-85fa-b58f88311938 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:34.472272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d791a32-ab2b-47e6-85fa-b58f88311938 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:34.473568 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 595/1193] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:01:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:01:34.478674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c6f59-1e30-4836-8a41-1e607b072ff4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:34.479089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c6f59-1e30-4836-8a41-1e607b072ff4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:34.479209 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1c6f59-1e30-4836-8a41-1e607b072ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:34.497176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4a1c6f59-1e30-4836-8a41-1e607b072ff4 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:01:34.498363 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4a1c6f59-1e30-4836-8a41-1e607b072ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:01:34.564062 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 599/1194] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:01:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:01:34.831058 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d791a32-ab2b-47e6-85fa-b58f88311938 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:34.913616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:34.935584 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d791a32-ab2b-47e6-85fa-b58f88311938 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:34.968787 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce8220-91b2-461a-9876-d69334ea3466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:34.969427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce8220-91b2-461a-9876-d69334ea3466 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:34.969702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce8220-91b2-461a-9876-d69334ea3466 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:34.969970 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce8220-91b2-461a-9876-d69334ea3466 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:34.979617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ce8220-91b2-461a-9876-d69334ea3466 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:01:34.984370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-81ce8220-91b2-461a-9876-d69334ea3466 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:01:34.984370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:01:34.984370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:01:34.984370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:01:34.984370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:01:34.984370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:01:34.984370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:01:34.984370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:01:34.984370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:01:34.984370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:01:34.984370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:01:34.984370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:01:34.984370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:01:34.984370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:01:34.984370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:01:34.984370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:01:34.985487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:01:34.985487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:01:34.985487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:01:34.985487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:01:34.985487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:01:34.985487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:01:34.985487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:01:34.985487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:01:34.985487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:01:34.985487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:01:34.985487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:01:34.985487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:01:34.985487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:01:34.985487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:01:34.985487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:01:34.985487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:01:34.986688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:01:34.986688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:01:34.986688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:01:34.986688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:01:34.986688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:01:34.986688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:01:34.986688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:01:34.986688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:01:34.986688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:01:34.986688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:01:34.986688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:01:34.986688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:01:34.986688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 596/1195] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:01:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:01:34.991572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c6f59-1e30-4836-8a41-1e607b072ff4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:35.057330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:35.088572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1c6f59-1e30-4836-8a41-1e607b072ff4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:35.121736 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3eb11-2cb9-47a6-9aba-a24f5749825b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:35.122329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3eb11-2cb9-47a6-9aba-a24f5749825b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:35.122605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3eb11-2cb9-47a6-9aba-a24f5749825b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:35.122882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3eb11-2cb9-47a6-9aba-a24f5749825b None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:35.123533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3eb11-2cb9-47a6-9aba-a24f5749825b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:01:35.123832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3eb11-2cb9-47a6-9aba-a24f5749825b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:35.125481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3eb11-2cb9-47a6-9aba-a24f5749825b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:35.133274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 600/1196] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:01:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:35.139471 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce8220-91b2-461a-9876-d69334ea3466 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:35.239836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce8220-91b2-461a-9876-d69334ea3466 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:35.272703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7aefbf-7874-40fb-8e07-d3df95410a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:35.273349 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aefbf-7874-40fb-8e07-d3df95410a49 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:35.273626 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aefbf-7874-40fb-8e07-d3df95410a49 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:35.273890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7aefbf-7874-40fb-8e07-d3df95410a49 None admin] PATH_INFO: `/v3/users/22aea11508614a0c8a681be2566fe898` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:35.274998 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7aefbf-7874-40fb-8e07-d3df95410a49 None admin] RBAC: Authorizing `identity:get_user(user_id=22aea11508614a0c8a681be2566fe898)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:35.276398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7aefbf-7874-40fb-8e07-d3df95410a49 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:35.278480 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 597/1197] 10.222.0.65 () {56 vars in 1177 bytes} [Wed Aug 31 19:01:35 2022] GET /identity/v3/users/22aea11508614a0c8a681be2566fe898 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:35.284378 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3eb11-2cb9-47a6-9aba-a24f5749825b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:35.381428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3eb11-2cb9-47a6-9aba-a24f5749825b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:35.409131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:35.414844 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f57dfd-516a-4853-9118-793a244a34d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:35.415431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f57dfd-516a-4853-9118-793a244a34d7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:35.415707 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f57dfd-516a-4853-9118-793a244a34d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:35.415970 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f57dfd-516a-4853-9118-793a244a34d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:35.479817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f57dfd-516a-4853-9118-793a244a34d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:35.481871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f57dfd-516a-4853-9118-793a244a34d7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:35.575161 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 601/1198] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:01:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:01:35.580935 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7aefbf-7874-40fb-8e07-d3df95410a49 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:35.591370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:35.677244 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7aefbf-7874-40fb-8e07-d3df95410a49 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:35.710493 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d42de0-71b4-4205-8e0d-7953b2eef3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:35.711112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d42de0-71b4-4205-8e0d-7953b2eef3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:35.711391 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d42de0-71b4-4205-8e0d-7953b2eef3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:35.711662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d42de0-71b4-4205-8e0d-7953b2eef3d4 None admin] PATH_INFO: `/v3/projects/3d7ffaaf899d4d1ba2bf4117c014b39d` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:35.712834 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d42de0-71b4-4205-8e0d-7953b2eef3d4 None admin] RBAC: Authorizing `identity:get_project(project_id=3d7ffaaf899d4d1ba2bf4117c014b39d)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:35.714333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d42de0-71b4-4205-8e0d-7953b2eef3d4 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:35.716049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 598/1199] 10.222.0.65 () {56 vars in 1186 bytes} [Wed Aug 31 19:01:35 2022] GET /identity/v3/projects/3d7ffaaf899d4d1ba2bf4117c014b39d => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:35.722132 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f57dfd-516a-4853-9118-793a244a34d7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:01:35.752825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:35.820938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f57dfd-516a-4853-9118-793a244a34d7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:01:35.853213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b42565-2050-445b-926a-8a486fc30c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:01:35.853824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b42565-2050-445b-926a-8a486fc30c9c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:01:35.854099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b42565-2050-445b-926a-8a486fc30c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:01:35.854369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b42565-2050-445b-926a-8a486fc30c9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:01:35.855082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b42565-2050-445b-926a-8a486fc30c9c None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=3d7ffaaf899d4d1ba2bf4117c014b39d, user.id=22aea11508614a0c8a681be2566fe898) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:01:35.855387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b42565-2050-445b-926a-8a486fc30c9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:01:35.857349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b42565-2050-445b-926a-8a486fc30c9c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:01:35.862866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 602/1200] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:01:35 2022] GET /identity/v3/role_assignments?user.id=22aea11508614a0c8a681be2566fe898&scope.project.id=3d7ffaaf899d4d1ba2bf4117c014b39d&role.id=3580c894dd974d35bc95848b8ba22598 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:01:45.705388 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=62 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:01:45.860355 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=52 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:31.135620 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8a009-4ded-4685-91f9-d9e5b1fb2562 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:31.136864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8a009-4ded-4685-91f9-d9e5b1fb2562 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:31.136864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de8a009-4ded-4685-91f9-d9e5b1fb2562 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:31.142886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 599/1201] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:02:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:02:31.144462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f4f6b-7159-4907-965f-06a11dd41bc1 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:31.144893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f4f6b-7159-4907-965f-06a11dd41bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:31.145086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10f4f6b-7159-4907-965f-06a11dd41bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:31.167333 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c10f4f6b-7159-4907-965f-06a11dd41bc1 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:02:31.169274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c10f4f6b-7159-4907-965f-06a11dd41bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:02:31.273392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 603/1202] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:02:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:02:31.572810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8a009-4ded-4685-91f9-d9e5b1fb2562 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:31.705698 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de8a009-4ded-4685-91f9-d9e5b1fb2562 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:31.748285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3c0a6-a9eb-488a-98d2-6da37c9fd9e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:31.749235 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3c0a6-a9eb-488a-98d2-6da37c9fd9e9 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:31.749512 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3c0a6-a9eb-488a-98d2-6da37c9fd9e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:31.749779 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a3c0a6-a9eb-488a-98d2-6da37c9fd9e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:31.761008 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a3c0a6-a9eb-488a-98d2-6da37c9fd9e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:31.763215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a3c0a6-a9eb-488a-98d2-6da37c9fd9e9 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:31.780466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-f2a3c0a6-a9eb-488a-98d2-6da37c9fd9e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:31.780466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:31.780466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:31.780466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:31.780466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:31.780466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:31.780466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:31.780466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:31.780466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:31.780466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:31.780466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:31.780466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:31.780466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:02:31.780466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:02:31.780466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:02:31.780466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:31.781736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:02:31.782705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:02:31.782705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:31.782705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:02:31.782705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 600/1203] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:31.793220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f4f6b-7159-4907-965f-06a11dd41bc1 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:31.953517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10f4f6b-7159-4907-965f-06a11dd41bc1 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:31.987779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:31.996221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9071298b-f2a8-4b59-b81c-858ad49e34c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:31.996945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071298b-f2a8-4b59-b81c-858ad49e34c8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:31.997312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071298b-f2a8-4b59-b81c-858ad49e34c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:31.997593 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9071298b-f2a8-4b59-b81c-858ad49e34c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:31.998347 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071298b-f2a8-4b59-b81c-858ad49e34c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:31.998628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071298b-f2a8-4b59-b81c-858ad49e34c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:32.000533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9071298b-f2a8-4b59-b81c-858ad49e34c8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:32.017244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 604/1204] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:32.021006 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3c0a6-a9eb-488a-98d2-6da37c9fd9e9 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:32.089834 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:32.145229 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a3c0a6-a9eb-488a-98d2-6da37c9fd9e9 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:32.187368 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4257f7-9b57-4839-8353-d4ed0102696b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:32.187965 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4257f7-9b57-4839-8353-d4ed0102696b None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:32.188239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4257f7-9b57-4839-8353-d4ed0102696b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:32.188503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4257f7-9b57-4839-8353-d4ed0102696b None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:32.189139 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4257f7-9b57-4839-8353-d4ed0102696b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:32.192837 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4257f7-9b57-4839-8353-d4ed0102696b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:32.229326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 601/1205] 10.222.0.65 () {60 vars in 1132 bytes} [Wed Aug 31 19:02:32 2022] POST /identity/v3/users => generated 251 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:02:34.404833 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0a58a-7ac1-4eb6-81af-748cc25cc6b1 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:34.405439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0a58a-7ac1-4eb6-81af-748cc25cc6b1 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:34.405439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0a58a-7ac1-4eb6-81af-748cc25cc6b1 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:34.406875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 605/1206] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:02:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:02:34.412136 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009d439-be54-4ae1-9cf1-76058094ec10 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:34.412625 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009d439-be54-4ae1-9cf1-76058094ec10 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:34.412717 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009d439-be54-4ae1-9cf1-76058094ec10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:34.432208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4009d439-be54-4ae1-9cf1-76058094ec10 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:02:34.434057 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4009d439-be54-4ae1-9cf1-76058094ec10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:02:34.511586 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 602/1207] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:02:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:02:34.846411 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0a58a-7ac1-4eb6-81af-748cc25cc6b1 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:34.968469 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0a58a-7ac1-4eb6-81af-748cc25cc6b1 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:35.008296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcff995-17cc-4882-b8b4-1f3e83a91fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:35.009299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcff995-17cc-4882-b8b4-1f3e83a91fef None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:35.012756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcff995-17cc-4882-b8b4-1f3e83a91fef None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:35.013235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcff995-17cc-4882-b8b4-1f3e83a91fef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:35.025778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcff995-17cc-4882-b8b4-1f3e83a91fef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:02:35.032405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-8dcff995-17cc-4882-b8b4-1f3e83a91fef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:02:35.032405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:35.032405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:35.032405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:35.032405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:35.032405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:35.032405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:35.032405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:35.032405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:35.032405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:35.032405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:35.032405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:35.032405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:02:35.032405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:02:35.032405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:02:35.032405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:02:35.033467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:02:35.033467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:02:35.033467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:02:35.033467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:02:35.033467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:35.033467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:35.033467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:35.033467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:35.033467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:35.033467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:35.033467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:35.033467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:35.033467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:35.033467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:35.033467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:35.033467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:35.034327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:35.034327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:35.034327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:35.034327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:35.034327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:02:35.034327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:02:35.034327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:02:35.034327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:02:35.034327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:02:35.034327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:02:35.034327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:02:35.034327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:02:35.034327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 606/1208] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 19:02:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:35.042050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009d439-be54-4ae1-9cf1-76058094ec10 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:35.091032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:35.166198 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009d439-be54-4ae1-9cf1-76058094ec10 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:35.205990 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4b948-af8a-4759-a770-cb7b00966d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:35.206695 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4b948-af8a-4759-a770-cb7b00966d04 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:35.206981 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4b948-af8a-4759-a770-cb7b00966d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:35.207264 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4b948-af8a-4759-a770-cb7b00966d04 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:35.208096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4b948-af8a-4759-a770-cb7b00966d04 None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:35.208440 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4b948-af8a-4759-a770-cb7b00966d04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:35.209865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4b948-af8a-4759-a770-cb7b00966d04 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:35.224451 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 603/1209] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 19:02:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:35.226079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcff995-17cc-4882-b8b4-1f3e83a91fef None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:35.290295 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:35.355390 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcff995-17cc-4882-b8b4-1f3e83a91fef None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:35.398507 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4349f5-02b9-473a-bd7a-eca43e29cdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:35.399150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4349f5-02b9-473a-bd7a-eca43e29cdf7 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:35.399432 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4349f5-02b9-473a-bd7a-eca43e29cdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:35.399833 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4349f5-02b9-473a-bd7a-eca43e29cdf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:35.411733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4349f5-02b9-473a-bd7a-eca43e29cdf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:35.413171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4349f5-02b9-473a-bd7a-eca43e29cdf7 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:35.430377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-dc4349f5-02b9-473a-bd7a-eca43e29cdf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:35.430377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:35.430377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:35.430377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:35.430377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:35.430377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:35.430377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:35.430377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:35.430377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:35.430377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:35.430377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:35.430377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:35.430377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:02:35.430377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:02:35.430377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:02:35.430377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:35.431384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:02:35.432602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:02:35.432602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:35.432602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:02:35.432602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 607/1210] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:35.437805 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4b948-af8a-4759-a770-cb7b00966d04 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:35.556321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:35.560182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4b948-af8a-4759-a770-cb7b00966d04 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:35.600433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815228cd-5e12-4bf4-acf7-2f787392734b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:35.601021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815228cd-5e12-4bf4-acf7-2f787392734b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:35.601343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815228cd-5e12-4bf4-acf7-2f787392734b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:35.601599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815228cd-5e12-4bf4-acf7-2f787392734b None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:35.602247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815228cd-5e12-4bf4-acf7-2f787392734b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:35.602551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815228cd-5e12-4bf4-acf7-2f787392734b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:35.603896 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815228cd-5e12-4bf4-acf7-2f787392734b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:35.621423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 604/1211] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:35.623444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4349f5-02b9-473a-bd7a-eca43e29cdf7 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:35.735584 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4349f5-02b9-473a-bd7a-eca43e29cdf7 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:35.737436 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:35.781357 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86998d36-7ef6-4071-8e99-9e5a557fc38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:35.782345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86998d36-7ef6-4071-8e99-9e5a557fc38c None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:35.782814 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86998d36-7ef6-4071-8e99-9e5a557fc38c None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:35.783233 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86998d36-7ef6-4071-8e99-9e5a557fc38c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:35.792460 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86998d36-7ef6-4071-8e99-9e5a557fc38c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:35.794021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86998d36-7ef6-4071-8e99-9e5a557fc38c None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:35.808284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-86998d36-7ef6-4071-8e99-9e5a557fc38c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:02:35.808284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:35.808284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:35.808284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:35.808284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:35.808284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:35.808284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:35.808284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:35.808284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:35.808284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:35.808284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:35.808284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:35.808284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:02:35.808284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:02:35.808284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:02:35.808284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:35.809836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:35.810790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:35.810790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:35.810790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:02:35.810790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:02:35.810790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:02:35.810790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:02:35.810790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:02:35.810790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:02:35.810790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:02:35.810790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:02:35.810790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 608/1212] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:35.816157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815228cd-5e12-4bf4-acf7-2f787392734b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:35.932143 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815228cd-5e12-4bf4-acf7-2f787392734b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:35.970382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5c336-7d1a-4f7f-b32a-d5508f81532a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:35.971012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5c336-7d1a-4f7f-b32a-d5508f81532a None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:35.971293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5c336-7d1a-4f7f-b32a-d5508f81532a None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:35.971564 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5c336-7d1a-4f7f-b32a-d5508f81532a None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:35.972326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5c336-7d1a-4f7f-b32a-d5508f81532a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:35.972627 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5c336-7d1a-4f7f-b32a-d5508f81532a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:35.974091 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5c336-7d1a-4f7f-b32a-d5508f81532a None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:35.992439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 605/1213] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 31 19:02:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:35.999408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86998d36-7ef6-4071-8e99-9e5a557fc38c None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:36.118329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86998d36-7ef6-4071-8e99-9e5a557fc38c None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:36.155603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598335e-3fb1-41a1-be48-c1d3677a527a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:36.156330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598335e-3fb1-41a1-be48-c1d3677a527a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:36.156498 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598335e-3fb1-41a1-be48-c1d3677a527a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:36.156754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c598335e-3fb1-41a1-be48-c1d3677a527a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:36.190338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:36.244894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598335e-3fb1-41a1-be48-c1d3677a527a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:36.247025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c598335e-3fb1-41a1-be48-c1d3677a527a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:36.382022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 609/1214] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:02:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:02:36.387388 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5c336-7d1a-4f7f-b32a-d5508f81532a None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:36.407005 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:36.493799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5c336-7d1a-4f7f-b32a-d5508f81532a None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:36.529360 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa1b5e-aefc-417c-95d2-5be47ecbb243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:36.529962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa1b5e-aefc-417c-95d2-5be47ecbb243 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:36.530242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa1b5e-aefc-417c-95d2-5be47ecbb243 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:36.530513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17aa1b5e-aefc-417c-95d2-5be47ecbb243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:36.540864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa1b5e-aefc-417c-95d2-5be47ecbb243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:36.542953 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17aa1b5e-aefc-417c-95d2-5be47ecbb243 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:36.559001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-17aa1b5e-aefc-417c-95d2-5be47ecbb243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:36.559001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:36.559001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:36.559001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:36.559001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:36.559001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:36.559001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:36.559001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:36.559001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:36.559001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:36.559001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:36.559001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:36.559001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:02:36.559001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:02:36.559001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:02:36.559001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:36.560051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:02:36.560987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:02:36.560987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:36.560987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:02:36.560987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 606/1215] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:02:36.566145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598335e-3fb1-41a1-be48-c1d3677a527a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:36.635602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:36.675778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c598335e-3fb1-41a1-be48-c1d3677a527a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:36.710983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a815d6e1-1e76-42aa-ba68-73e5fd00087f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:36.711682 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a815d6e1-1e76-42aa-ba68-73e5fd00087f None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:36.712032 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a815d6e1-1e76-42aa-ba68-73e5fd00087f None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:36.712375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a815d6e1-1e76-42aa-ba68-73e5fd00087f None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:36.713103 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a815d6e1-1e76-42aa-ba68-73e5fd00087f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:36.713481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a815d6e1-1e76-42aa-ba68-73e5fd00087f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:36.715432 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a815d6e1-1e76-42aa-ba68-73e5fd00087f None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:36.727268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 610/1216] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:36.733690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa1b5e-aefc-417c-95d2-5be47ecbb243 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:36.822276 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:36.863660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17aa1b5e-aefc-417c-95d2-5be47ecbb243 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:36.913076 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22176815-e6ae-4f90-a64e-a5abd3129540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:36.913993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22176815-e6ae-4f90-a64e-a5abd3129540 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:36.914405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22176815-e6ae-4f90-a64e-a5abd3129540 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:36.914817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22176815-e6ae-4f90-a64e-a5abd3129540 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:36.925485 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22176815-e6ae-4f90-a64e-a5abd3129540 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:36.927040 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22176815-e6ae-4f90-a64e-a5abd3129540 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:36.942246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-22176815-e6ae-4f90-a64e-a5abd3129540 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:02:36.942246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:36.942246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:36.942246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:36.942246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:36.942246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:36.942246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:36.942246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:36.942246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:36.942246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:36.942246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:36.942246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:36.942246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:02:36.942246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:02:36.942246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:36.943489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:36.944405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:02:36.944405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:02:36.944405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:02:36.944405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:02:36.944405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:02:36.944405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:02:36.944405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:02:36.944405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:02:36.944405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 607/1217] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:02:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:36.949388 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a815d6e1-1e76-42aa-ba68-73e5fd00087f None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:37.058236 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a815d6e1-1e76-42aa-ba68-73e5fd00087f None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:37.085539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:37.101463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd791e-9f6b-4336-ad35-62033cfba9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:37.102071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd791e-9f6b-4336-ad35-62033cfba9db None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:37.102351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd791e-9f6b-4336-ad35-62033cfba9db None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:37.102627 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd791e-9f6b-4336-ad35-62033cfba9db None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:37.103317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd791e-9f6b-4336-ad35-62033cfba9db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:37.103621 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd791e-9f6b-4336-ad35-62033cfba9db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:37.105092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd791e-9f6b-4336-ad35-62033cfba9db None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:37.117726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 611/1218] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:02:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:37.124159 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22176815-e6ae-4f90-a64e-a5abd3129540 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:37.254400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22176815-e6ae-4f90-a64e-a5abd3129540 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:37.287257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:37.293868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd6bc12-5045-459a-8250-0465d6b79195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:37.294564 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6bc12-5045-459a-8250-0465d6b79195 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:37.294898 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6bc12-5045-459a-8250-0465d6b79195 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:37.295237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd6bc12-5045-459a-8250-0465d6b79195 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:37.296035 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd6bc12-5045-459a-8250-0465d6b79195 None admin] RBAC: Adding query filter params (role.id=6e93fc836fe74c9daa11c7b2848a2922, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=74f1846e395e4d5483fcbb41eb28a77d) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:37.296416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd6bc12-5045-459a-8250-0465d6b79195 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:37.297882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd6bc12-5045-459a-8250-0465d6b79195 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:37.304314 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 608/1219] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:02:37 2022] GET /identity/v3/role_assignments?user.id=74f1846e395e4d5483fcbb41eb28a77d&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=6e93fc836fe74c9daa11c7b2848a2922 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:02:39.306272 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919edf8e-3ffa-4510-80a4-c86ff0d15716 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:39.306802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919edf8e-3ffa-4510-80a4-c86ff0d15716 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:39.306802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919edf8e-3ffa-4510-80a4-c86ff0d15716 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:39.308999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 612/1220] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:02:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:02:39.314027 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3620848-2dca-4443-8ed9-254ce3b3927e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:39.314455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3620848-2dca-4443-8ed9-254ce3b3927e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:39.314609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3620848-2dca-4443-8ed9-254ce3b3927e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:39.332505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d3620848-2dca-4443-8ed9-254ce3b3927e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:02:39.333728 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d3620848-2dca-4443-8ed9-254ce3b3927e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:02:39.405079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 609/1221] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:02:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:02:39.674073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919edf8e-3ffa-4510-80a4-c86ff0d15716 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:39.792028 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919edf8e-3ffa-4510-80a4-c86ff0d15716 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:39.826414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04401c0-09ce-4c89-b136-c32b21a564e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:39.827022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04401c0-09ce-4c89-b136-c32b21a564e5 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:39.827300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04401c0-09ce-4c89-b136-c32b21a564e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:39.827573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04401c0-09ce-4c89-b136-c32b21a564e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:39.838149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04401c0-09ce-4c89-b136-c32b21a564e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:02:39.843636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-f04401c0-09ce-4c89-b136-c32b21a564e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:02:39.843636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:39.843636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:39.843636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:39.843636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:39.843636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:39.843636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:39.843636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:39.843636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:39.843636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:39.843636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:39.843636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:39.843636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:02:39.843636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:02:39.843636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:02:39.843636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:02:39.844832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:02:39.844832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:02:39.844832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:02:39.844832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:02:39.844832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:39.844832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:39.844832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:39.844832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:39.844832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:39.844832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:39.844832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:39.844832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:39.844832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:39.844832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:39.844832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:39.844832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:39.845891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:39.845891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:39.845891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:39.845891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:39.845891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:02:39.845891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:02:39.845891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:02:39.845891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:02:39.845891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:02:39.845891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:02:39.845891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:02:39.845891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:02:39.845891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 613/1222] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 19:02:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:39.850842 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3620848-2dca-4443-8ed9-254ce3b3927e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:39.961709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3620848-2dca-4443-8ed9-254ce3b3927e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:39.986020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:39.997067 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753596c-3b93-44f7-b0da-1c71ec2fc70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:39.997700 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753596c-3b93-44f7-b0da-1c71ec2fc70c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:39.997977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753596c-3b93-44f7-b0da-1c71ec2fc70c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:39.998247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1753596c-3b93-44f7-b0da-1c71ec2fc70c None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:39.999153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753596c-3b93-44f7-b0da-1c71ec2fc70c None admin] RBAC: Adding query filter params (name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:39.999452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753596c-3b93-44f7-b0da-1c71ec2fc70c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:40.000815 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1753596c-3b93-44f7-b0da-1c71ec2fc70c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:40.008848 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 610/1223] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 19:02:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:40.015381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04401c0-09ce-4c89-b136-c32b21a564e5 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:40.066765 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:40.128812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04401c0-09ce-4c89-b136-c32b21a564e5 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:40.165712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195054b6-edbc-4e37-8c58-6d8af4c09913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:40.166438 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195054b6-edbc-4e37-8c58-6d8af4c09913 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:40.166754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195054b6-edbc-4e37-8c58-6d8af4c09913 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:40.167092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195054b6-edbc-4e37-8c58-6d8af4c09913 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:40.179299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195054b6-edbc-4e37-8c58-6d8af4c09913 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:40.180888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195054b6-edbc-4e37-8c58-6d8af4c09913 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:40.199185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-195054b6-edbc-4e37-8c58-6d8af4c09913 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:40.199185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:40.199185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:40.199185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:40.199185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:40.199185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:40.199185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:40.199185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:40.199185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:40.199185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:40.199185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:40.199185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:40.199185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:02:40.199185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:02:40.199185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:02:40.199185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:40.200345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:02:40.201497 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:02:40.201497 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:40.201497 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:02:40.202889 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 614/1224] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:40.208812 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753596c-3b93-44f7-b0da-1c71ec2fc70c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:40.314964 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1753596c-3b93-44f7-b0da-1c71ec2fc70c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:40.352959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3743121-8b25-417b-9d68-0433fcdcd633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:40.353598 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3743121-8b25-417b-9d68-0433fcdcd633 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:40.353873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3743121-8b25-417b-9d68-0433fcdcd633 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:40.354144 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3743121-8b25-417b-9d68-0433fcdcd633 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:40.355093 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3743121-8b25-417b-9d68-0433fcdcd633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:40.355396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3743121-8b25-417b-9d68-0433fcdcd633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:40.356776 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3743121-8b25-417b-9d68-0433fcdcd633 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:40.368919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 611/1225] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:40.375416 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195054b6-edbc-4e37-8c58-6d8af4c09913 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:40.482544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:40.490150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195054b6-edbc-4e37-8c58-6d8af4c09913 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:40.527586 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e1183-0db3-4c3b-8b0f-33cb2917a096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:40.528269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e1183-0db3-4c3b-8b0f-33cb2917a096 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:40.528538 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e1183-0db3-4c3b-8b0f-33cb2917a096 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:40.528804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e1183-0db3-4c3b-8b0f-33cb2917a096 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:40.536798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e1183-0db3-4c3b-8b0f-33cb2917a096 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:40.538333 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e1183-0db3-4c3b-8b0f-33cb2917a096 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:40.551400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-e69e1183-0db3-4c3b-8b0f-33cb2917a096 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:02:40.551400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:40.551400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:40.551400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:40.551400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:40.551400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:40.551400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:40.551400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:40.551400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:40.551400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:40.551400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:40.551400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:40.551400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:02:40.551400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:02:40.551400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:02:40.551400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:40.552590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:40.553605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:40.553605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:40.553605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:02:40.553605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:02:40.553605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:02:40.553605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:02:40.553605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:02:40.553605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:02:40.553605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:02:40.553605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:02:40.553605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 615/1226] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:02:40.560103 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3743121-8b25-417b-9d68-0433fcdcd633 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:40.614893 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:40.672324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3743121-8b25-417b-9d68-0433fcdcd633 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:40.708348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffb382-d5a7-4749-807b-f48aeb7ec891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:40.708939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffb382-d5a7-4749-807b-f48aeb7ec891 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:40.709231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffb382-d5a7-4749-807b-f48aeb7ec891 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:40.709503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffb382-d5a7-4749-807b-f48aeb7ec891 None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:40.710216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffb382-d5a7-4749-807b-f48aeb7ec891 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:40.710514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffb382-d5a7-4749-807b-f48aeb7ec891 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:40.711960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffb382-d5a7-4749-807b-f48aeb7ec891 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:40.731041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 612/1227] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 31 19:02:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:40.737072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e1183-0db3-4c3b-8b0f-33cb2917a096 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:40.850779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e1183-0db3-4c3b-8b0f-33cb2917a096 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:40.885660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c662a8d-a005-4457-bde6-90c14adaef3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:40.886726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c662a8d-a005-4457-bde6-90c14adaef3a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:40.886998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c662a8d-a005-4457-bde6-90c14adaef3a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:40.887327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c662a8d-a005-4457-bde6-90c14adaef3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:40.899173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c662a8d-a005-4457-bde6-90c14adaef3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:40.900605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c662a8d-a005-4457-bde6-90c14adaef3a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:40.915283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-9c662a8d-a005-4457-bde6-90c14adaef3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:40.915283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:40.915283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:40.915283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:40.915283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:40.915283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:40.915283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:40.915283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:40.915283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:40.915283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:40.915283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:40.915283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:40.915283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:02:40.915283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:02:40.915283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:02:40.915283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:40.922244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:02:40.923439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:02:40.923439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:40.923439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:02:40.923439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 616/1228] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:40.923712 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffb382-d5a7-4749-807b-f48aeb7ec891 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:41.026148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffb382-d5a7-4749-807b-f48aeb7ec891 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:41.029022 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:41.061603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841d5d4-8eef-423b-9a6a-60d521d47c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:41.062232 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841d5d4-8eef-423b-9a6a-60d521d47c70 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:41.062485 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841d5d4-8eef-423b-9a6a-60d521d47c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:41.062746 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7841d5d4-8eef-423b-9a6a-60d521d47c70 None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:41.063368 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841d5d4-8eef-423b-9a6a-60d521d47c70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:41.063682 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841d5d4-8eef-423b-9a6a-60d521d47c70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:41.065010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7841d5d4-8eef-423b-9a6a-60d521d47c70 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:41.077042 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 613/1229] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:41.083320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c662a8d-a005-4457-bde6-90c14adaef3a None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:41.095537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:41.211465 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c662a8d-a005-4457-bde6-90c14adaef3a None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:41.251332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa4d67-d69b-487b-b04f-1e2f0c30823d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:41.251955 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa4d67-d69b-487b-b04f-1e2f0c30823d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:41.252231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa4d67-d69b-487b-b04f-1e2f0c30823d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:41.252534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa4d67-d69b-487b-b04f-1e2f0c30823d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:41.259290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa4d67-d69b-487b-b04f-1e2f0c30823d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:41.260724 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fa4d67-d69b-487b-b04f-1e2f0c30823d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:41.273239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-87fa4d67-d69b-487b-b04f-1e2f0c30823d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:02:41.273239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:41.273239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:41.273239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:41.273239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:41.273239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:41.273239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:41.273239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:41.273239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:41.273239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:41.273239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:41.273239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:41.273239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:02:41.273239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:02:41.273239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:41.274324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:41.275311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:02:41.275311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:02:41.275311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:02:41.275311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:02:41.275311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:02:41.275311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:02:41.275311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:02:41.275311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:02:41.275311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 617/1230] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:02:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:41.280513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841d5d4-8eef-423b-9a6a-60d521d47c70 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:41.384730 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7841d5d4-8eef-423b-9a6a-60d521d47c70 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:41.421569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ebf16-8a9b-4625-bb67-317ef7baf7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:41.422166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ebf16-8a9b-4625-bb67-317ef7baf7db None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:41.422441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ebf16-8a9b-4625-bb67-317ef7baf7db None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:41.422707 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ebf16-8a9b-4625-bb67-317ef7baf7db None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:41.423380 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ebf16-8a9b-4625-bb67-317ef7baf7db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:41.423787 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ebf16-8a9b-4625-bb67-317ef7baf7db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:41.425527 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ebf16-8a9b-4625-bb67-317ef7baf7db None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:41.438682 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 614/1231] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:02:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:02:41.445393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa4d67-d69b-487b-b04f-1e2f0c30823d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:41.514707 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:41.550685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa4d67-d69b-487b-b04f-1e2f0c30823d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:41.594243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df2fe0-4bb5-4a6e-b6b6-7d3221c26219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:41.594930 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df2fe0-4bb5-4a6e-b6b6-7d3221c26219 None admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:41.595230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df2fe0-4bb5-4a6e-b6b6-7d3221c26219 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:41.595545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df2fe0-4bb5-4a6e-b6b6-7d3221c26219 None admin] PATH_INFO: `/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/74f1846e395e4d5483fcbb41eb28a77d/roles/6e93fc836fe74c9daa11c7b2848a2922` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:41.614449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df2fe0-4bb5-4a6e-b6b6-7d3221c26219 None admin] RBAC: Authorizing `identity:create_grant(project_id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user_id=74f1846e395e4d5483fcbb41eb28a77d, role_id=6e93fc836fe74c9daa11c7b2848a2922)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:41.616195 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df2fe0-4bb5-4a6e-b6b6-7d3221c26219 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:41.629904 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 618/1232] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 19:02:41 2022] PUT /identity/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/74f1846e395e4d5483fcbb41eb28a77d/roles/6e93fc836fe74c9daa11c7b2848a2922 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:02:43.627338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdddb67-8a42-4edb-b33a-1bf02bf582c6 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:43.627891 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdddb67-8a42-4edb-b33a-1bf02bf582c6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:43.627891 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdddb67-8a42-4edb-b33a-1bf02bf582c6 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:43.629384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 615/1233] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:02:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:02:43.635196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddfdff-e2d2-489d-a892-01992174d10c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:43.635626 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddfdff-e2d2-489d-a892-01992174d10c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:43.635717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddfdff-e2d2-489d-a892-01992174d10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:43.654890 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-98ddfdff-e2d2-489d-a892-01992174d10c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:02:43.656232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-98ddfdff-e2d2-489d-a892-01992174d10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:02:43.730610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 619/1234] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:02:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:02:44.030376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdddb67-8a42-4edb-b33a-1bf02bf582c6 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:44.058700 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:44.147251 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdddb67-8a42-4edb-b33a-1bf02bf582c6 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:44.182199 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a879bf-062b-4b54-af5f-6bceb77f255d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:44.182920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a879bf-062b-4b54-af5f-6bceb77f255d None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:44.183202 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a879bf-062b-4b54-af5f-6bceb77f255d None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:44.183497 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a879bf-062b-4b54-af5f-6bceb77f255d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:44.200372 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a879bf-062b-4b54-af5f-6bceb77f255d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:02:44.205206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-c1a879bf-062b-4b54-af5f-6bceb77f255d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:02:44.205206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:44.205206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:44.205206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:44.205206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:44.205206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:44.205206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:44.205206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:44.205206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:44.205206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:44.205206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:44.205206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:44.205206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:02:44.205206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:02:44.205206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:02:44.205206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:02:44.206458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:02:44.206458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:02:44.206458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:02:44.206458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:02:44.206458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:44.206458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:44.206458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:44.206458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:44.206458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:44.206458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:44.206458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:44.206458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:44.206458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:44.206458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:44.206458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:44.206458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:44.207308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:44.207308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:44.207308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:44.207308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:44.207308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:02:44.207308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:02:44.207308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:02:44.207308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:02:44.207308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:02:44.207308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:02:44.207308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 31 19:02:44.207308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:02:44.207308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 616/1235] 10.222.0.65 () {56 vars in 1102 bytes} [Wed Aug 31 19:02:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:44.215077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddfdff-e2d2-489d-a892-01992174d10c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:44.264626 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:44.325226 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ddfdff-e2d2-489d-a892-01992174d10c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:44.363910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5c54c-49cc-40f5-ba42-045cb1f7b079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:44.364528 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5c54c-49cc-40f5-ba42-045cb1f7b079 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:44.364802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5c54c-49cc-40f5-ba42-045cb1f7b079 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:44.365067 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f5c54c-49cc-40f5-ba42-045cb1f7b079 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:44.365750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5c54c-49cc-40f5-ba42-045cb1f7b079 None admin] RBAC: Adding query filter params (name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:44.366043 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5c54c-49cc-40f5-ba42-045cb1f7b079 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:44.367389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f5c54c-49cc-40f5-ba42-045cb1f7b079 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:44.376008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 620/1236] 10.222.0.65 () {56 vars in 1103 bytes} [Wed Aug 31 19:02:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:44.382014 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a879bf-062b-4b54-af5f-6bceb77f255d None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:44.474295 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:44.491786 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a879bf-062b-4b54-af5f-6bceb77f255d None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:44.529368 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486597e-57b4-45ab-914a-38da5e71e7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:44.530100 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486597e-57b4-45ab-914a-38da5e71e7fb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:44.530379 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486597e-57b4-45ab-914a-38da5e71e7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:44.530663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486597e-57b4-45ab-914a-38da5e71e7fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:44.542150 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486597e-57b4-45ab-914a-38da5e71e7fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:44.543653 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486597e-57b4-45ab-914a-38da5e71e7fb None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:44.558820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-3486597e-57b4-45ab-914a-38da5e71e7fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:44.558820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:44.558820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:44.558820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:44.558820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:44.558820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:44.558820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:44.558820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:44.558820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:44.558820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:44.558820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:44.558820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:44.558820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:02:44.558820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:02:44.558820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:02:44.558820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:44.560407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:02:44.562078 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:02:44.562078 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:44.562078 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:02:44.562078 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 617/1237] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:44.566014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5c54c-49cc-40f5-ba42-045cb1f7b079 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:44.687374 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f5c54c-49cc-40f5-ba42-045cb1f7b079 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:44.699355 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:44.730855 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877221d9-d8fe-449e-a635-1c108c6d724b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:44.731474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877221d9-d8fe-449e-a635-1c108c6d724b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:44.731754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877221d9-d8fe-449e-a635-1c108c6d724b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:44.732023 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877221d9-d8fe-449e-a635-1c108c6d724b None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:44.732660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877221d9-d8fe-449e-a635-1c108c6d724b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:44.732961 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877221d9-d8fe-449e-a635-1c108c6d724b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:44.734362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877221d9-d8fe-449e-a635-1c108c6d724b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:44.746844 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 621/1238] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:02:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:44.753223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486597e-57b4-45ab-914a-38da5e71e7fb None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:44.862426 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486597e-57b4-45ab-914a-38da5e71e7fb None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:44.900484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83677b0b-761e-4e3b-9d4f-2becdb464c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:44.901117 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83677b0b-761e-4e3b-9d4f-2becdb464c92 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:44.901439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83677b0b-761e-4e3b-9d4f-2becdb464c92 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:44.901716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83677b0b-761e-4e3b-9d4f-2becdb464c92 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:44.904786 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:44.910872 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83677b0b-761e-4e3b-9d4f-2becdb464c92 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:44.912420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83677b0b-761e-4e3b-9d4f-2becdb464c92 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:44.924556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-83677b0b-761e-4e3b-9d4f-2becdb464c92 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:02:44.924556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:44.924556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:44.924556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:44.924556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:44.924556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:44.924556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:44.924556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:44.924556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:44.924556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:44.924556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:44.924556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:44.924556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:02:44.924556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:02:44.924556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:02:44.924556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:44.929745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:44.931602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:44.931602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:44.931602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:02:44.931602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:02:44.931602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:02:44.931602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:02:44.931602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:02:44.931602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:02:44.931602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:02:44.931602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:02:44.931602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 618/1239] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:44.933231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877221d9-d8fe-449e-a635-1c108c6d724b None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:45.041519 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877221d9-d8fe-449e-a635-1c108c6d724b None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:45.083421 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdc8ad-0af1-4138-bb9d-426e5062f35d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:45.084011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdc8ad-0af1-4138-bb9d-426e5062f35d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:45.084298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdc8ad-0af1-4138-bb9d-426e5062f35d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:45.084669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdc8ad-0af1-4138-bb9d-426e5062f35d None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:45.085381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdc8ad-0af1-4138-bb9d-426e5062f35d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:45.085682 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdc8ad-0af1-4138-bb9d-426e5062f35d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:45.087096 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdc8ad-0af1-4138-bb9d-426e5062f35d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:45.107777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 622/1240] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 31 19:02:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:45.113205 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83677b0b-761e-4e3b-9d4f-2becdb464c92 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:45.229294 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83677b0b-761e-4e3b-9d4f-2becdb464c92 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:45.270621 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e975bb-9d0a-4069-be02-bc8c8df4e4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:45.271234 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e975bb-9d0a-4069-be02-bc8c8df4e4ee None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:45.271511 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e975bb-9d0a-4069-be02-bc8c8df4e4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:45.271779 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e975bb-9d0a-4069-be02-bc8c8df4e4ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:45.340605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:45.348674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e975bb-9d0a-4069-be02-bc8c8df4e4ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:45.350392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e975bb-9d0a-4069-be02-bc8c8df4e4ee None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:45.458367 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 619/1241] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:02:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:02:45.464259 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdc8ad-0af1-4138-bb9d-426e5062f35d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:45.496210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:45.574692 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdc8ad-0af1-4138-bb9d-426e5062f35d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:45.616492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac6683-afd4-4e65-a1af-133331dc3893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:45.617049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac6683-afd4-4e65-a1af-133331dc3893 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:45.617351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac6683-afd4-4e65-a1af-133331dc3893 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:45.617618 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac6683-afd4-4e65-a1af-133331dc3893 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:45.629028 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac6683-afd4-4e65-a1af-133331dc3893 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:45.631215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac6683-afd4-4e65-a1af-133331dc3893 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:45.652415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-b6ac6683-afd4-4e65-a1af-133331dc3893 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:45.652415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:45.652415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:45.652415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:45.652415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:45.652415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:45.652415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:45.652415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:45.652415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:45.652415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:45.652415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:45.652415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:45.652415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:02:45.652415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:02:45.652415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:02:45.652415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:45.653829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:02:45.654919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:02:45.654919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:45.654919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:02:45.654919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 623/1242] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:45.663252 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e975bb-9d0a-4069-be02-bc8c8df4e4ee None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:45.771572 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:45.778981 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e975bb-9d0a-4069-be02-bc8c8df4e4ee None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:45.816456 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da2b36-0d08-4d8f-a691-7b1bcfa43c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:45.817089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da2b36-0d08-4d8f-a691-7b1bcfa43c7b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:45.817393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da2b36-0d08-4d8f-a691-7b1bcfa43c7b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:45.817666 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92da2b36-0d08-4d8f-a691-7b1bcfa43c7b None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:45.818310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da2b36-0d08-4d8f-a691-7b1bcfa43c7b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:45.818604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da2b36-0d08-4d8f-a691-7b1bcfa43c7b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:45.819971 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92da2b36-0d08-4d8f-a691-7b1bcfa43c7b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:45.832066 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 620/1243] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:02:45.838487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac6683-afd4-4e65-a1af-133331dc3893 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:45.924193 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:45.948667 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac6683-afd4-4e65-a1af-133331dc3893 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:45.986925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab39264-b924-439b-a727-a43077b64792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:45.987558 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab39264-b924-439b-a727-a43077b64792 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:45.987838 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab39264-b924-439b-a727-a43077b64792 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:45.988114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab39264-b924-439b-a727-a43077b64792 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:45.996158 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab39264-b924-439b-a727-a43077b64792 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:45.997658 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab39264-b924-439b-a727-a43077b64792 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:46.009783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-bab39264-b924-439b-a727-a43077b64792 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:02:46.009783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:46.009783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:46.009783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:46.009783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:46.009783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:46.009783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:46.009783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:46.009783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:46.009783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:46.009783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:46.009783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:46.009783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:02:46.009783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:02:46.009783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:46.016585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:46.017610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:02:46.017610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:02:46.017610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:02:46.017610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:02:46.017610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:02:46.017610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:02:46.017610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:02:46.017610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:02:46.017610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 624/1244] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:02:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 31 19:02:46.018103 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da2b36-0d08-4d8f-a691-7b1bcfa43c7b None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:46.127195 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92da2b36-0d08-4d8f-a691-7b1bcfa43c7b None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:46.162832 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9170fbf-ddc2-46ab-b55f-f0b95b8fda27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:46.163434 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9170fbf-ddc2-46ab-b55f-f0b95b8fda27 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:46.163710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9170fbf-ddc2-46ab-b55f-f0b95b8fda27 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:46.163985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9170fbf-ddc2-46ab-b55f-f0b95b8fda27 None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:46.164675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9170fbf-ddc2-46ab-b55f-f0b95b8fda27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:46.164967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9170fbf-ddc2-46ab-b55f-f0b95b8fda27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:46.166409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9170fbf-ddc2-46ab-b55f-f0b95b8fda27 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:46.179452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 621/1245] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:02:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:46.185717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab39264-b924-439b-a727-a43077b64792 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:46.293220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab39264-b924-439b-a727-a43077b64792 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:46.328539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3df132-74dc-4beb-ab30-1f64d757786b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:46.329145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3df132-74dc-4beb-ab30-1f64d757786b None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:46.329448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3df132-74dc-4beb-ab30-1f64d757786b None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:46.329717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3df132-74dc-4beb-ab30-1f64d757786b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:46.330467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3df132-74dc-4beb-ab30-1f64d757786b None admin] RBAC: Adding query filter params (role.id=6e93fc836fe74c9daa11c7b2848a2922, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=74f1846e395e4d5483fcbb41eb28a77d) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:46.330744 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3df132-74dc-4beb-ab30-1f64d757786b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:46.332191 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3df132-74dc-4beb-ab30-1f64d757786b None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:46.353217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 625/1246] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:02:46 2022] GET /identity/v3/role_assignments?user.id=74f1846e395e4d5483fcbb41eb28a77d&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=6e93fc836fe74c9daa11c7b2848a2922 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:48.390009 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430a06c-a0dc-4fde-9db2-96c8ab3df6f8 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:48.390564 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430a06c-a0dc-4fde-9db2-96c8ab3df6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:48.390564 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430a06c-a0dc-4fde-9db2-96c8ab3df6f8 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:48.391939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 622/1247] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:02:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:02:48.397479 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9ebe3-179c-4f15-b7d9-6369bf4f13d2 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:48.397980 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9ebe3-179c-4f15-b7d9-6369bf4f13d2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:48.398076 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e9ebe3-179c-4f15-b7d9-6369bf4f13d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:48.404898 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:48.418021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b5e9ebe3-179c-4f15-b7d9-6369bf4f13d2 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:02:48.419232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b5e9ebe3-179c-4f15-b7d9-6369bf4f13d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:02:48.494330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 626/1248] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:02:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:02:48.845020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430a06c-a0dc-4fde-9db2-96c8ab3df6f8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:48.864188 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:48.968109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430a06c-a0dc-4fde-9db2-96c8ab3df6f8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:49.011870 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7069bc-00fc-420f-98c9-7f6bcd2d9fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:49.012504 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7069bc-00fc-420f-98c9-7f6bcd2d9fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:49.012781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7069bc-00fc-420f-98c9-7f6bcd2d9fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:49.013053 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7069bc-00fc-420f-98c9-7f6bcd2d9fe7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:49.023664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7069bc-00fc-420f-98c9-7f6bcd2d9fe7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:02:49.028939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-8a7069bc-00fc-420f-98c9-7f6bcd2d9fe7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:02:49.028939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:49.028939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:49.028939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:49.028939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:49.028939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:49.028939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:49.028939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:49.028939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:49.028939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:49.028939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:49.028939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:49.028939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:02:49.028939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:02:49.028939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:02:49.028939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:02:49.030119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:02:49.030119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:02:49.030119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:02:49.030119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:02:49.030119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:49.030119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:49.030119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:49.030119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:49.030119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:49.030119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:49.030119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:49.030119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:49.030119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:49.030119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:49.030119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:49.030119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:49.030975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:49.030975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:49.030975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:49.030975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:49.030975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:02:49.030975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:02:49.030975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:02:49.030975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:02:49.030975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:02:49.030975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:02:49.030975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:02:49.030975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:02:49.030975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 623/1249] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:02:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:49.036257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9ebe3-179c-4f15-b7d9-6369bf4f13d2 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:49.141277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e9ebe3-179c-4f15-b7d9-6369bf4f13d2 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:49.172912 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:49.178907 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31608b0c-e537-453f-a4b1-93fa3eec6763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:49.179529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31608b0c-e537-453f-a4b1-93fa3eec6763 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:49.179806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31608b0c-e537-453f-a4b1-93fa3eec6763 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:49.180073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31608b0c-e537-453f-a4b1-93fa3eec6763 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:49.180797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31608b0c-e537-453f-a4b1-93fa3eec6763 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:49.181095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31608b0c-e537-453f-a4b1-93fa3eec6763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:49.182497 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31608b0c-e537-453f-a4b1-93fa3eec6763 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:49.191002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 627/1250] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:02:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:49.197055 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7069bc-00fc-420f-98c9-7f6bcd2d9fe7 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:49.283143 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:49.310061 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7069bc-00fc-420f-98c9-7f6bcd2d9fe7 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:49.349609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a79afd-419e-40f7-90be-673e2fd44d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:49.350379 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a79afd-419e-40f7-90be-673e2fd44d21 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:49.350696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a79afd-419e-40f7-90be-673e2fd44d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:49.351036 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a79afd-419e-40f7-90be-673e2fd44d21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:49.362285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a79afd-419e-40f7-90be-673e2fd44d21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:49.364792 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a79afd-419e-40f7-90be-673e2fd44d21 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:49.383120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-62a79afd-419e-40f7-90be-673e2fd44d21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:49.383120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:49.383120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:49.383120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:49.383120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:49.383120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:49.383120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:49.383120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:49.383120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:49.383120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:49.383120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:49.383120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:49.383120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:02:49.383120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:02:49.383120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:02:49.383120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:49.384340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:02:49.385310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:02:49.385310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:49.385310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:02:49.389875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 624/1251] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:49.391166 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31608b0c-e537-453f-a4b1-93fa3eec6763 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:49.506623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31608b0c-e537-453f-a4b1-93fa3eec6763 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:49.545585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d5963-1be8-45b6-ae78-2be5e3abc777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:49.546200 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d5963-1be8-45b6-ae78-2be5e3abc777 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:49.546479 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d5963-1be8-45b6-ae78-2be5e3abc777 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:49.546750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d5963-1be8-45b6-ae78-2be5e3abc777 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:49.547389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d5963-1be8-45b6-ae78-2be5e3abc777 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:49.547689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d5963-1be8-45b6-ae78-2be5e3abc777 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:49.549047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d5963-1be8-45b6-ae78-2be5e3abc777 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:49.564245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 628/1252] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:49.570800 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a79afd-419e-40f7-90be-673e2fd44d21 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:49.685199 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a79afd-419e-40f7-90be-673e2fd44d21 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:49.712370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:49.720344 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8b787-4494-4d9c-8f3a-9d613c7643b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:49.720985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8b787-4494-4d9c-8f3a-9d613c7643b2 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:49.721266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8b787-4494-4d9c-8f3a-9d613c7643b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:49.721562 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8b787-4494-4d9c-8f3a-9d613c7643b2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:49.729071 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8b787-4494-4d9c-8f3a-9d613c7643b2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:49.730624 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8b787-4494-4d9c-8f3a-9d613c7643b2 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:49.741999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-1aa8b787-4494-4d9c-8f3a-9d613c7643b2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:02:49.741999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:49.741999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:49.741999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:49.741999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:49.741999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:49.741999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:49.741999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:49.741999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:49.741999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:49.741999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:49.741999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:49.741999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:02:49.741999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:02:49.741999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:02:49.741999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:49.743303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:49.744259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:49.744259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:49.744259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:02:49.744259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:02:49.744259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:02:49.744259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:02:49.744259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:02:49.744259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:02:49.744259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:02:49.744259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:02:49.744259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 625/1253] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:49.749906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d5963-1be8-45b6-ae78-2be5e3abc777 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:49.813068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:49.857448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d5963-1be8-45b6-ae78-2be5e3abc777 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:49.894528 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4a109-cd74-428a-bee2-1cb2d4e1b085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:49.895224 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4a109-cd74-428a-bee2-1cb2d4e1b085 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:49.895502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4a109-cd74-428a-bee2-1cb2d4e1b085 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:49.895773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4a109-cd74-428a-bee2-1cb2d4e1b085 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:49.896860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4a109-cd74-428a-bee2-1cb2d4e1b085 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:49.897145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4a109-cd74-428a-bee2-1cb2d4e1b085 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:49.898835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4a109-cd74-428a-bee2-1cb2d4e1b085 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:49.916652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 629/1254] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 31 19:02:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:49.923387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8b787-4494-4d9c-8f3a-9d613c7643b2 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:50.028082 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8b787-4494-4d9c-8f3a-9d613c7643b2 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:50.064631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93027a5b-9dba-4309-89a2-067199f23cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:50.065161 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93027a5b-9dba-4309-89a2-067199f23cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:50.065472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93027a5b-9dba-4309-89a2-067199f23cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:50.065738 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93027a5b-9dba-4309-89a2-067199f23cd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:50.125418 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:50.138350 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93027a5b-9dba-4309-89a2-067199f23cd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:50.139781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93027a5b-9dba-4309-89a2-067199f23cd0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:50.246082 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 626/1255] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:02:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:02:50.252063 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4a109-cd74-428a-bee2-1cb2d4e1b085 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:50.360859 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4a109-cd74-428a-bee2-1cb2d4e1b085 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:50.377947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:50.402618 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99170982-826d-46aa-b165-66dc718ce3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:50.403384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99170982-826d-46aa-b165-66dc718ce3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:50.403706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99170982-826d-46aa-b165-66dc718ce3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:50.404013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99170982-826d-46aa-b165-66dc718ce3e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:50.415591 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99170982-826d-46aa-b165-66dc718ce3e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:50.417248 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99170982-826d-46aa-b165-66dc718ce3e8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:50.433581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-99170982-826d-46aa-b165-66dc718ce3e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:50.433581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:50.433581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:50.433581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:50.433581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:50.433581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:50.433581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:50.433581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:50.433581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:50.433581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:50.433581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:50.433581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:50.433581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:02:50.433581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:02:50.433581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:02:50.433581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:50.434733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:02:50.435642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:02:50.435642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:50.435642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:02:50.435642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 630/1256] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:50.440758 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93027a5b-9dba-4309-89a2-067199f23cd0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:50.539882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:50.553793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93027a5b-9dba-4309-89a2-067199f23cd0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:50.592629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e22554-279a-4021-a978-0ac5686701cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:50.593473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e22554-279a-4021-a978-0ac5686701cd None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:50.593820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e22554-279a-4021-a978-0ac5686701cd None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:50.594192 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e22554-279a-4021-a978-0ac5686701cd None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:50.595032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e22554-279a-4021-a978-0ac5686701cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:50.595328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e22554-279a-4021-a978-0ac5686701cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:50.597145 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e22554-279a-4021-a978-0ac5686701cd None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:50.610704 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 627/1257] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:02:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:50.617077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99170982-826d-46aa-b165-66dc718ce3e8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:50.732412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99170982-826d-46aa-b165-66dc718ce3e8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:50.768511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962918cd-2e06-4332-bc86-674287a9cbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:50.769147 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962918cd-2e06-4332-bc86-674287a9cbcb None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:50.769472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962918cd-2e06-4332-bc86-674287a9cbcb None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:50.769737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962918cd-2e06-4332-bc86-674287a9cbcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:50.778309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962918cd-2e06-4332-bc86-674287a9cbcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:50.779768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962918cd-2e06-4332-bc86-674287a9cbcb None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:50.793463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-962918cd-2e06-4332-bc86-674287a9cbcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:02:50.793463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:50.793463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:50.793463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:50.793463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:50.793463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:50.793463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:50.793463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:50.793463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:50.793463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:50.793463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:50.793463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:50.793463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:02:50.793463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:02:50.793463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:50.794635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:50.795572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:02:50.795572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:02:50.795572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:02:50.795572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:02:50.795572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:02:50.795572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:02:50.795572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:02:50.795572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:02:50.795572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 631/1258] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:02:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:50.800918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e22554-279a-4021-a978-0ac5686701cd None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:50.905721 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e22554-279a-4021-a978-0ac5686701cd None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:50.942419 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851c92b-2ced-4431-9345-b4e50ac5372b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:50.943038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851c92b-2ced-4431-9345-b4e50ac5372b None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:50.943316 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851c92b-2ced-4431-9345-b4e50ac5372b None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:50.943586 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851c92b-2ced-4431-9345-b4e50ac5372b None admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:50.944285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851c92b-2ced-4431-9345-b4e50ac5372b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:50.944589 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851c92b-2ced-4431-9345-b4e50ac5372b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:50.946053 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851c92b-2ced-4431-9345-b4e50ac5372b None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:50.964549 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 628/1259] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:02:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:50.965782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962918cd-2e06-4332-bc86-674287a9cbcb None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:50.993691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:51.070084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962918cd-2e06-4332-bc86-674287a9cbcb None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:51.103315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b013866b-1e33-42d5-a059-5dd20ceb9de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:51.103899 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013866b-1e33-42d5-a059-5dd20ceb9de9 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:51.104173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013866b-1e33-42d5-a059-5dd20ceb9de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:51.104449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b013866b-1e33-42d5-a059-5dd20ceb9de9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:51.105709 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013866b-1e33-42d5-a059-5dd20ceb9de9 None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=74f1846e395e4d5483fcbb41eb28a77d) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:51.106009 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013866b-1e33-42d5-a059-5dd20ceb9de9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:51.107408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b013866b-1e33-42d5-a059-5dd20ceb9de9 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:51.112941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 632/1260] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:02:50 2022] GET /identity/v3/role_assignments?user.id=74f1846e395e4d5483fcbb41eb28a77d&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=3580c894dd974d35bc95848b8ba22598 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:53.002040 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe663a4d-6b4c-4edb-b58d-5cf6435207ab None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:53.002599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe663a4d-6b4c-4edb-b58d-5cf6435207ab None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:53.002599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe663a4d-6b4c-4edb-b58d-5cf6435207ab None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:53.003915 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 629/1261] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:02:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:02:53.008848 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9b9e8-4caa-4b72-9c4d-0f78e568d077 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:53.009255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9b9e8-4caa-4b72-9c4d-0f78e568d077 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:53.009424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf9b9e8-4caa-4b72-9c4d-0f78e568d077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:53.027006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3cf9b9e8-4caa-4b72-9c4d-0f78e568d077 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:02:53.028172 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3cf9b9e8-4caa-4b72-9c4d-0f78e568d077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:02:53.091460 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 633/1262] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:02:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:02:53.361630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe663a4d-6b4c-4edb-b58d-5cf6435207ab None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:53.372363 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:53.461742 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe663a4d-6b4c-4edb-b58d-5cf6435207ab None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:53.494949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f79c3-fa08-436e-9b7a-7fa62a990ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:53.496210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f79c3-fa08-436e-9b7a-7fa62a990ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:53.496487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f79c3-fa08-436e-9b7a-7fa62a990ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:53.496753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50f79c3-fa08-436e-9b7a-7fa62a990ed6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:53.506759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50f79c3-fa08-436e-9b7a-7fa62a990ed6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:02:53.512069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-a50f79c3-fa08-436e-9b7a-7fa62a990ed6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:02:53.512069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:53.512069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:53.512069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:53.512069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:53.512069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:53.512069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:53.512069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:53.512069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:53.512069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:53.512069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:53.512069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:53.512069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:02:53.512069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:02:53.512069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:02:53.512069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:02:53.513166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:02:53.513166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:02:53.513166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:02:53.513166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:02:53.513166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:53.513166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:53.513166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:53.513166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:53.513166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:53.513166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:53.513166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:53.513166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:53.513166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:53.513166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:53.513166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:53.513166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:53.514242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:53.514242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:53.514242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:53.514242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:53.514242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:02:53.514242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:02:53.514242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:02:53.514242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:02:53.514242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:02:53.514242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:02:53.514242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:02:53.514242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:02:53.514242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 630/1263] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:02:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:53.518990 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9b9e8-4caa-4b72-9c4d-0f78e568d077 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:53.536482 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:53.631253 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf9b9e8-4caa-4b72-9c4d-0f78e568d077 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:53.663604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd74c0b-6780-4c03-95c0-4ed43bf36c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:53.664197 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd74c0b-6780-4c03-95c0-4ed43bf36c87 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:53.664473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd74c0b-6780-4c03-95c0-4ed43bf36c87 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:53.664742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd74c0b-6780-4c03-95c0-4ed43bf36c87 None admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:53.665420 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd74c0b-6780-4c03-95c0-4ed43bf36c87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:53.665718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd74c0b-6780-4c03-95c0-4ed43bf36c87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:53.667582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd74c0b-6780-4c03-95c0-4ed43bf36c87 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:53.675668 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 634/1264] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:02:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:53.681433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f79c3-fa08-436e-9b7a-7fa62a990ed6 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:53.746688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:53.779052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50f79c3-fa08-436e-9b7a-7fa62a990ed6 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:53.812716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835785c-7939-4bd4-819c-e0fce6be2552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:53.813363 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835785c-7939-4bd4-819c-e0fce6be2552 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:53.813641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835785c-7939-4bd4-819c-e0fce6be2552 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:53.813909 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c835785c-7939-4bd4-819c-e0fce6be2552 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:53.823668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835785c-7939-4bd4-819c-e0fce6be2552 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:53.825105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c835785c-7939-4bd4-819c-e0fce6be2552 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:53.839399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-c835785c-7939-4bd4-819c-e0fce6be2552 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:53.839399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:53.839399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:53.839399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:53.839399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:53.839399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:53.839399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:53.839399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:53.839399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:53.839399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:53.839399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:53.839399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:53.839399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:02:53.839399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:02:53.839399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:02:53.839399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:53.840509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:02:53.841601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:02:53.841601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:53.841601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:02:53.841601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 631/1265] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:53.846316 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd74c0b-6780-4c03-95c0-4ed43bf36c87 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:53.925651 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:53.944426 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd74c0b-6780-4c03-95c0-4ed43bf36c87 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:53.976355 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25994b-0831-41f3-ab59-ea83e6aa1e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:53.976942 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25994b-0831-41f3-ab59-ea83e6aa1e38 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:53.977238 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25994b-0831-41f3-ab59-ea83e6aa1e38 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:53.977502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c25994b-0831-41f3-ab59-ea83e6aa1e38 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:53.978128 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25994b-0831-41f3-ab59-ea83e6aa1e38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:53.978422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25994b-0831-41f3-ab59-ea83e6aa1e38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:53.980321 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c25994b-0831-41f3-ab59-ea83e6aa1e38 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:53.991553 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 635/1266] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:53.997279 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835785c-7939-4bd4-819c-e0fce6be2552 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:54.097397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c835785c-7939-4bd4-819c-e0fce6be2552 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:54.116778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:54.131047 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ad53d-adfe-4564-aea0-8fcf2dc3aeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:54.131664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ad53d-adfe-4564-aea0-8fcf2dc3aeb1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:54.131941 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ad53d-adfe-4564-aea0-8fcf2dc3aeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:54.132210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287ad53d-adfe-4564-aea0-8fcf2dc3aeb1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:54.139453 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ad53d-adfe-4564-aea0-8fcf2dc3aeb1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:54.140856 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287ad53d-adfe-4564-aea0-8fcf2dc3aeb1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:54.152121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-287ad53d-adfe-4564-aea0-8fcf2dc3aeb1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:02:54.152121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:54.152121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:54.152121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:54.152121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:54.152121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:54.152121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:54.152121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:54.152121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:54.152121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:54.152121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:54.152121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:54.152121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:02:54.152121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:02:54.152121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:02:54.152121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:54.153258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:54.154583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:54.154583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:54.154583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:02:54.154583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:02:54.154583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:02:54.154583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:02:54.154583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:02:54.154583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:02:54.154583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:02:54.154583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:02:54.154583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 632/1267] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:53 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:54.159114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25994b-0831-41f3-ab59-ea83e6aa1e38 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:54.257558 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c25994b-0831-41f3-ab59-ea83e6aa1e38 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:54.290391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd47dad-38cc-4c25-81b7-90d29205eda8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:54.290980 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd47dad-38cc-4c25-81b7-90d29205eda8 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:54.291256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd47dad-38cc-4c25-81b7-90d29205eda8 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:54.291524 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd47dad-38cc-4c25-81b7-90d29205eda8 None admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:54.292783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd47dad-38cc-4c25-81b7-90d29205eda8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:54.293081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd47dad-38cc-4c25-81b7-90d29205eda8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:54.294530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd47dad-38cc-4c25-81b7-90d29205eda8 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:54.302904 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:54.311375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 636/1268] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 31 19:02:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:54.317020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ad53d-adfe-4564-aea0-8fcf2dc3aeb1 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:54.438476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287ad53d-adfe-4564-aea0-8fcf2dc3aeb1 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:54.473293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddeb18-04b2-4955-a76f-9fdf666fc341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:54.473848 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddeb18-04b2-4955-a76f-9fdf666fc341 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:54.474123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddeb18-04b2-4955-a76f-9fdf666fc341 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:54.474390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ddeb18-04b2-4955-a76f-9fdf666fc341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:54.484654 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ddeb18-04b2-4955-a76f-9fdf666fc341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:54.486213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ddeb18-04b2-4955-a76f-9fdf666fc341 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:54.500032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-58ddeb18-04b2-4955-a76f-9fdf666fc341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:54.500032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:54.500032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:54.500032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:54.500032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:54.500032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:54.500032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:54.500032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:54.500032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:54.500032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:54.500032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:54.500032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:54.500032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:02:54.500032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:02:54.500032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:02:54.500032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:54.501072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:02:54.502231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:02:54.502231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:54.502231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:02:54.502231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 633/1269] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:54.507240 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd47dad-38cc-4c25-81b7-90d29205eda8 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:54.613036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd47dad-38cc-4c25-81b7-90d29205eda8 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:54.646644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119d070-ede7-4e47-8eb4-b855d51d5878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:54.647232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119d070-ede7-4e47-8eb4-b855d51d5878 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:54.647502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119d070-ede7-4e47-8eb4-b855d51d5878 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:54.647766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b119d070-ede7-4e47-8eb4-b855d51d5878 None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:54.648384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119d070-ede7-4e47-8eb4-b855d51d5878 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:54.648676 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119d070-ede7-4e47-8eb4-b855d51d5878 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:54.650037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b119d070-ede7-4e47-8eb4-b855d51d5878 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:54.660810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 637/1270] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:02:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:54.666504 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddeb18-04b2-4955-a76f-9fdf666fc341 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:54.698584 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:54.765007 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ddeb18-04b2-4955-a76f-9fdf666fc341 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:54.797038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df504de-57fa-4837-bff6-db91af1380d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:54.797670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df504de-57fa-4837-bff6-db91af1380d8 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:54.797947 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df504de-57fa-4837-bff6-db91af1380d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:54.798210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df504de-57fa-4837-bff6-db91af1380d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:54.804549 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df504de-57fa-4837-bff6-db91af1380d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:54.805995 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df504de-57fa-4837-bff6-db91af1380d8 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:54.817387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-0df504de-57fa-4837-bff6-db91af1380d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:02:54.817387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:54.817387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:54.817387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:54.817387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:54.817387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:54.817387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:54.817387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:54.817387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:54.817387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:54.817387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:54.817387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:54.817387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:02:54.817387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:02:54.817387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:54.818411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:54.819723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:02:54.819723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:02:54.819723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:02:54.819723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:02:54.819723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:02:54.819723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:02:54.819723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:02:54.819723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:02:54.819723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 634/1271] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:02:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:54.824766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119d070-ede7-4e47-8eb4-b855d51d5878 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:54.881080 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:54.921331 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b119d070-ede7-4e47-8eb4-b855d51d5878 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:54.954942 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61c2cbc-ce30-4489-b452-305428217226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:54.955535 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c2cbc-ce30-4489-b452-305428217226 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:54.955814 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c2cbc-ce30-4489-b452-305428217226 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:54.956084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61c2cbc-ce30-4489-b452-305428217226 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:54.956773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c2cbc-ce30-4489-b452-305428217226 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:54.957073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c2cbc-ce30-4489-b452-305428217226 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:54.958514 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61c2cbc-ce30-4489-b452-305428217226 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:54.970120 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 638/1272] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:02:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:54.976018 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df504de-57fa-4837-bff6-db91af1380d8 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:55.064867 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:55.074134 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df504de-57fa-4837-bff6-db91af1380d8 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:55.106434 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266ebb0b-6ed3-460f-9ab2-7e2dec41f586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:55.107068 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ebb0b-6ed3-460f-9ab2-7e2dec41f586 None admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:55.107344 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ebb0b-6ed3-460f-9ab2-7e2dec41f586 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:55.107615 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266ebb0b-6ed3-460f-9ab2-7e2dec41f586 None admin] PATH_INFO: `/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/74f1846e395e4d5483fcbb41eb28a77d/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:55.109479 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ebb0b-6ed3-460f-9ab2-7e2dec41f586 None admin] RBAC: Authorizing `identity:create_grant(project_id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user_id=74f1846e395e4d5483fcbb41eb28a77d, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:55.110880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266ebb0b-6ed3-460f-9ab2-7e2dec41f586 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:55.123815 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 635/1273] 10.222.0.65 () {58 vars in 1439 bytes} [Wed Aug 31 19:02:54 2022] PUT /identity/v3/projects/fdc1fee7aedd4f4a9fba0fd6fb614e33/users/74f1846e395e4d5483fcbb41eb28a77d/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:02:57.070346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df19d6-8539-451c-a420-39dabb15ef56 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:57.070909 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df19d6-8539-451c-a420-39dabb15ef56 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:57.070909 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df19d6-8539-451c-a420-39dabb15ef56 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:57.072234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 639/1274] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:02:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:02:57.079402 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac25ed-1f79-4db6-91fb-bbeb545b71c2 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:57.079818 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac25ed-1f79-4db6-91fb-bbeb545b71c2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:57.080071 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac25ed-1f79-4db6-91fb-bbeb545b71c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:57.098162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-fbac25ed-1f79-4db6-91fb-bbeb545b71c2 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:02:57.099333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-fbac25ed-1f79-4db6-91fb-bbeb545b71c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:02:57.162863 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 636/1275] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:02:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:02:57.427759 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df19d6-8539-451c-a420-39dabb15ef56 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:57.528793 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df19d6-8539-451c-a420-39dabb15ef56 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:57.546834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:57.569539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea85165-f985-4c03-b3e2-9d90423132da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:57.570383 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea85165-f985-4c03-b3e2-9d90423132da None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:57.570708 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea85165-f985-4c03-b3e2-9d90423132da None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:57.571072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea85165-f985-4c03-b3e2-9d90423132da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:57.583266 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea85165-f985-4c03-b3e2-9d90423132da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 31 19:02:57.588757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-fea85165-f985-4c03-b3e2-9d90423132da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:02:57.588757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:57.588757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:57.588757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:57.588757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:57.588757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:57.588757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:57.588757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:57.588757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:57.588757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:57.588757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:57.588757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:57.588757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 31 19:02:57.588757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 31 19:02:57.588757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 31 19:02:57.588757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 31 19:02:57.590186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 31 19:02:57.590186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 31 19:02:57.590186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 31 19:02:57.590186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 31 19:02:57.590186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:57.590186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:57.590186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:57.590186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:57.590186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:57.590186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:57.590186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:57.590186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:57.590186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:57.590186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:57.590186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:57.590186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:57.591053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:57.591053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:57.591053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:57.591053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:57.591053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 31 19:02:57.591053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 31 19:02:57.591053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 31 19:02:57.591053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 31 19:02:57.591053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 31 19:02:57.591053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 31 19:02:57.591053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 31 19:02:57.591053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:02:57.591053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 640/1276] 10.222.0.65 () {56 vars in 1096 bytes} [Wed Aug 31 19:02:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:57.596065 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac25ed-1f79-4db6-91fb-bbeb545b71c2 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:57.675836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:57.694926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac25ed-1f79-4db6-91fb-bbeb545b71c2 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:57.727343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab45fe0-eebb-4899-9f11-4b73dbc0bef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:57.727935 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab45fe0-eebb-4899-9f11-4b73dbc0bef0 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:57.728211 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab45fe0-eebb-4899-9f11-4b73dbc0bef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:57.728504 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab45fe0-eebb-4899-9f11-4b73dbc0bef0 None admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:57.729151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab45fe0-eebb-4899-9f11-4b73dbc0bef0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:57.730124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab45fe0-eebb-4899-9f11-4b73dbc0bef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:57.731476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab45fe0-eebb-4899-9f11-4b73dbc0bef0 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:57.739162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 637/1277] 10.222.0.65 () {56 vars in 1099 bytes} [Wed Aug 31 19:02:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:57.744915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea85165-f985-4c03-b3e2-9d90423132da None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:57.841829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea85165-f985-4c03-b3e2-9d90423132da None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:57.874522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818f6bb-edbb-410b-8b16-b963463f96ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:57.875124 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818f6bb-edbb-410b-8b16-b963463f96ad None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:57.875396 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818f6bb-edbb-410b-8b16-b963463f96ad None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:57.875662 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0818f6bb-edbb-410b-8b16-b963463f96ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:57.885893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818f6bb-edbb-410b-8b16-b963463f96ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:57.887315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0818f6bb-edbb-410b-8b16-b963463f96ad None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:57.900852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-0818f6bb-edbb-410b-8b16-b963463f96ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:57.900852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:57.900852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:57.900852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:57.900852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:57.900852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:57.900852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:57.900852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:57.900852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:57.900852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:57.900852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:57.900852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:57.900852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:02:57.900852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:02:57.900852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:02:57.900852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:57.907015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:02:57.907996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:02:57.907996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:57.907996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:02:57.907996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 641/1278] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:57.908224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab45fe0-eebb-4899-9f11-4b73dbc0bef0 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:58.004645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab45fe0-eebb-4899-9f11-4b73dbc0bef0 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:58.036983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dde828-db50-420d-a132-e1905df6463c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:58.037592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dde828-db50-420d-a132-e1905df6463c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:58.037868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dde828-db50-420d-a132-e1905df6463c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:58.038134 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dde828-db50-420d-a132-e1905df6463c None admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:58.038756 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dde828-db50-420d-a132-e1905df6463c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:58.039054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dde828-db50-420d-a132-e1905df6463c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:58.040384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dde828-db50-420d-a132-e1905df6463c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:58.052231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 638/1279] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:02:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:02:58.057783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818f6bb-edbb-410b-8b16-b963463f96ad None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:58.088071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:58.156856 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0818f6bb-edbb-410b-8b16-b963463f96ad None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:58.189091 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60216a-5cb1-4a07-b473-3f52e35d9eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:58.189726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60216a-5cb1-4a07-b473-3f52e35d9eab None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:58.190004 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60216a-5cb1-4a07-b473-3f52e35d9eab None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:58.190275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b60216a-5cb1-4a07-b473-3f52e35d9eab None admin] PATH_INFO: `/v3/users/placement` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:58.197775 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60216a-5cb1-4a07-b473-3f52e35d9eab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:58.199746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b60216a-5cb1-4a07-b473-3f52e35d9eab None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:58.210655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-0b60216a-5cb1-4a07-b473-3f52e35d9eab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:02:58.210655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:58.210655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:58.210655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:58.210655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:58.210655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:58.210655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:58.210655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:58.210655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:58.210655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:58.210655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:58.210655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:58.210655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 31 19:02:58.210655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 31 19:02:58.210655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 31 19:02:58.210655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:58.211875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:58.213282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:58.213282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:58.213282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 31 19:02:58.213282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 31 19:02:58.213282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 31 19:02:58.213282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 31 19:02:58.213282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 31 19:02:58.213282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 31 19:02:58.213282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 31 19:02:58.213282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:02:58.213282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 642/1280] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:58.219091 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dde828-db50-420d-a132-e1905df6463c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:58.222886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:58.315497 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dde828-db50-420d-a132-e1905df6463c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:58.348387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d470796-0f3d-44bf-876a-dd3891a9ad1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:58.348974 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d470796-0f3d-44bf-876a-dd3891a9ad1f None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:58.349280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d470796-0f3d-44bf-876a-dd3891a9ad1f None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:58.349552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d470796-0f3d-44bf-876a-dd3891a9ad1f None admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:58.350263 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d470796-0f3d-44bf-876a-dd3891a9ad1f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:58.350562 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d470796-0f3d-44bf-876a-dd3891a9ad1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:58.351968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d470796-0f3d-44bf-876a-dd3891a9ad1f None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:58.368026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 639/1281] 10.222.0.65 () {56 vars in 1143 bytes} [Wed Aug 31 19:02:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:58.373983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60216a-5cb1-4a07-b473-3f52e35d9eab None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:58.462097 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:58.475328 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b60216a-5cb1-4a07-b473-3f52e35d9eab None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:58.508812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969bb38-afbe-49eb-8abc-bc89ccbaca86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:58.509424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969bb38-afbe-49eb-8abc-bc89ccbaca86 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:58.509696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969bb38-afbe-49eb-8abc-bc89ccbaca86 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:58.509959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8969bb38-afbe-49eb-8abc-bc89ccbaca86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:58.575614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969bb38-afbe-49eb-8abc-bc89ccbaca86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:58.576974 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8969bb38-afbe-49eb-8abc-bc89ccbaca86 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:58.660931 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:58.670938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 643/1282] 10.222.0.65 () {58 vars in 1282 bytes} [Wed Aug 31 19:02:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 31 19:02:58.676744 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d470796-0f3d-44bf-876a-dd3891a9ad1f None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:58.736095 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:58.773964 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d470796-0f3d-44bf-876a-dd3891a9ad1f None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:58.811215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef15baa-5978-4ecf-8570-93212192f59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:58.812069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef15baa-5978-4ecf-8570-93212192f59c None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:58.812346 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef15baa-5978-4ecf-8570-93212192f59c None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:58.812612 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef15baa-5978-4ecf-8570-93212192f59c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:58.822789 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef15baa-5978-4ecf-8570-93212192f59c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:58.824257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef15baa-5978-4ecf-8570-93212192f59c None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:58.840864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-3ef15baa-5978-4ecf-8570-93212192f59c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:58.840864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:58.840864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:58.840864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:58.840864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:58.840864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:58.840864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:58.840864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:58.840864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:58.840864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:58.840864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:58.840864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:58.840864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 31 19:02:58.840864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 31 19:02:58.840864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 31 19:02:58.840864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:58.841854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 31 19:02:58.843015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 31 19:02:58.843015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 31 19:02:58.843015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:02:58.843015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 640/1283] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:02:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:58.847658 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969bb38-afbe-49eb-8abc-bc89ccbaca86 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:58.943663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8969bb38-afbe-49eb-8abc-bc89ccbaca86 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:58.975477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5335c9-06c9-4951-92a7-10a7520b501d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:58.976058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5335c9-06c9-4951-92a7-10a7520b501d None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:58.976332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5335c9-06c9-4951-92a7-10a7520b501d None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:58.976598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5335c9-06c9-4951-92a7-10a7520b501d None admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:58.977240 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5335c9-06c9-4951-92a7-10a7520b501d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:58.977537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5335c9-06c9-4951-92a7-10a7520b501d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:58.978883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5335c9-06c9-4951-92a7-10a7520b501d None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:58.990184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 644/1284] 10.222.0.65 () {56 vars in 1109 bytes} [Wed Aug 31 19:02:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:58.996332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef15baa-5978-4ecf-8570-93212192f59c None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:59.091906 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef15baa-5978-4ecf-8570-93212192f59c None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:59.105654 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:59.126301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf693e85-c08a-43e3-99d8-1b6e3bff4609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:59.126926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf693e85-c08a-43e3-99d8-1b6e3bff4609 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:59.127203 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf693e85-c08a-43e3-99d8-1b6e3bff4609 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:59.127473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf693e85-c08a-43e3-99d8-1b6e3bff4609 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:59.133925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf693e85-c08a-43e3-99d8-1b6e3bff4609 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:59.135352 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf693e85-c08a-43e3-99d8-1b6e3bff4609 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:59.146278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application [None req-cf693e85-c08a-43e3-99d8-1b6e3bff4609 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:02:59.146278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:02:59.146278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:02:59.146278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:02:59.146278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:02:59.146278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:02:59.146278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:02:59.146278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:02:59.146278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:02:59.146278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:02:59.146278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:02:59.146278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:02:59.146278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 31 19:02:59.146278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 31 19:02:59.146278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application with Lock( Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:02:59.147287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:02:59.148538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 31 19:02:59.148538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 31 19:02:59.148538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 31 19:02:59.148538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 31 19:02:59.148538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 31 19:02:59.148538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 31 19:02:59.148538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 31 19:02:59.148538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: ERROR keystone.server.flask.application Aug 31 19:02:59.148538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 641/1285] 10.222.0.65 () {56 vars in 1111 bytes} [Wed Aug 31 19:02:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:02:59.153311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5335c9-06c9-4951-92a7-10a7520b501d None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:59.205366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:02:59.258263 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5335c9-06c9-4951-92a7-10a7520b501d None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:59.290036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c2077-3291-4222-a390-d16609c9f214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:59.290620 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c2077-3291-4222-a390-d16609c9f214 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:59.290891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c2077-3291-4222-a390-d16609c9f214 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:59.291154 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c2077-3291-4222-a390-d16609c9f214 None admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:59.292484 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c2077-3291-4222-a390-d16609c9f214 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:59.292784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c2077-3291-4222-a390-d16609c9f214 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:59.294217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c2077-3291-4222-a390-d16609c9f214 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:59.305596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 645/1286] 10.222.0.65 () {56 vars in 1148 bytes} [Wed Aug 31 19:02:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:02:59.311597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf693e85-c08a-43e3-99d8-1b6e3bff4609 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:02:59.408144 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf693e85-c08a-43e3-99d8-1b6e3bff4609 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:02:59.439861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62e771-fc4a-4ee9-a432-d26a052b7c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:02:59.440432 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62e771-fc4a-4ee9-a432-d26a052b7c80 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:02:59.440704 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62e771-fc4a-4ee9-a432-d26a052b7c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:02:59.440970 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62e771-fc4a-4ee9-a432-d26a052b7c80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:02:59.441702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62e771-fc4a-4ee9-a432-d26a052b7c80 None admin] RBAC: Adding query filter params (role.id=3580c894dd974d35bc95848b8ba22598, scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33, user.id=74f1846e395e4d5483fcbb41eb28a77d) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:02:59.442001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62e771-fc4a-4ee9-a432-d26a052b7c80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:02:59.443383 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62e771-fc4a-4ee9-a432-d26a052b7c80 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:02:59.448823 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 642/1287] 10.222.0.65 () {56 vars in 1374 bytes} [Wed Aug 31 19:02:59 2022] GET /identity/v3/role_assignments?user.id=74f1846e395e4d5483fcbb41eb28a77d&scope.project.id=fdc1fee7aedd4f4a9fba0fd6fb614e33&role.id=3580c894dd974d35bc95848b8ba22598 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:01.369686 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de1de1-19a9-4643-9f40-87683cb5d8ce None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:01.370805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de1de1-19a9-4643-9f40-87683cb5d8ce None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:01.370805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42de1de1-19a9-4643-9f40-87683cb5d8ce None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:01.372113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 646/1288] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:03:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:01.377633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f07f28-fe1f-44c8-b061-64ffb611f4a2 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:01.378040 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f07f28-fe1f-44c8-b061-64ffb611f4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:01.378176 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f07f28-fe1f-44c8-b061-64ffb611f4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:01.395755 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-55f07f28-fe1f-44c8-b061-64ffb611f4a2 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:01.396904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-55f07f28-fe1f-44c8-b061-64ffb611f4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:01.424993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:01.461556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 643/1289] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:03:01.751831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de1de1-19a9-4643-9f40-87683cb5d8ce None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:01.851414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42de1de1-19a9-4643-9f40-87683cb5d8ce None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:01.862782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:01.886688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf50b23-3cff-4101-8c07-82dc78db66db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:01.887296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf50b23-3cff-4101-8c07-82dc78db66db None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:01.887566 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf50b23-3cff-4101-8c07-82dc78db66db None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:01.887831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf50b23-3cff-4101-8c07-82dc78db66db None admin] PATH_INFO: `/v3/services/placement` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:01.888393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf50b23-3cff-4101-8c07-82dc78db66db None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:01.889788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf50b23-3cff-4101-8c07-82dc78db66db None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-7bf50b23-3cff-4101-8c07-82dc78db66db None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:01.898345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:01.899577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:01.899577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:01.899577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:01.899577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:01.899577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:01.899577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:01.899577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:03:01.899577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:03:01.899577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:03:01.899577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:03:01.899577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:01.899577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:01.899577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:01.899577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:01.899577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:01.899577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:01.900773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:01.900773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:01.900773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:01.900773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:01.900773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:01.900773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:01.900773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:01.900773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:01.900773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:01.900773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:01.900773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:03:01.900773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:03:01.900773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:03:01.900773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:03:01.900773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 647/1290] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 19:03:01 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:01.906064 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f07f28-fe1f-44c8-b061-64ffb611f4a2 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:02.000409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f07f28-fe1f-44c8-b061-64ffb611f4a2 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:02.033643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3718d9a-b8c1-4065-b361-d663f1b80ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:02.034167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3718d9a-b8c1-4065-b361-d663f1b80ebb None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:02.034439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3718d9a-b8c1-4065-b361-d663f1b80ebb None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:02.034704 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3718d9a-b8c1-4065-b361-d663f1b80ebb None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:02.035312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3718d9a-b8c1-4065-b361-d663f1b80ebb None admin] RBAC: Adding query filter params (name=placement) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:02.035603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3718d9a-b8c1-4065-b361-d663f1b80ebb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:02.036934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3718d9a-b8c1-4065-b361-d663f1b80ebb None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:02.041650 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 644/1291] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Aug 31 19:03:01 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:02.047402 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf50b23-3cff-4101-8c07-82dc78db66db None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:02.144597 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf50b23-3cff-4101-8c07-82dc78db66db None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:02.176713 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd25b45-b87d-4bbf-ae9c-57b066a5056a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:02.177325 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd25b45-b87d-4bbf-ae9c-57b066a5056a None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:02.177603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd25b45-b87d-4bbf-ae9c-57b066a5056a None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:02.177873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd25b45-b87d-4bbf-ae9c-57b066a5056a None admin] PATH_INFO: `/v3/services` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:02.178488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd25b45-b87d-4bbf-ae9c-57b066a5056a None admin] RBAC: Adding query filter params (type=placement) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:02.178787 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd25b45-b87d-4bbf-ae9c-57b066a5056a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:02.180136 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd25b45-b87d-4bbf-ae9c-57b066a5056a None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:02.184906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 648/1292] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Aug 31 19:03:02 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:04.047981 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ac136-0049-4d03-8f6c-18653eb56895 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:04.048546 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ac136-0049-4d03-8f6c-18653eb56895 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:04.048546 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ac136-0049-4d03-8f6c-18653eb56895 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:04.049875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 645/1293] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:03:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:04.054995 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661bc46-5982-4fa5-88af-1b6cfe8cf20e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:04.055414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661bc46-5982-4fa5-88af-1b6cfe8cf20e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:04.055540 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b661bc46-5982-4fa5-88af-1b6cfe8cf20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:04.073729 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b661bc46-5982-4fa5-88af-1b6cfe8cf20e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:04.075029 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b661bc46-5982-4fa5-88af-1b6cfe8cf20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:04.103780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:04.140520 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 649/1294] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:03:04.407644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ac136-0049-4d03-8f6c-18653eb56895 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:04.438755 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:04.507238 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef4ac136-0049-4d03-8f6c-18653eb56895 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:04.540117 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cceb874-8192-4aeb-8ff6-e799132711ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:04.540705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceb874-8192-4aeb-8ff6-e799132711ec None admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:04.540982 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceb874-8192-4aeb-8ff6-e799132711ec None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:04.541272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceb874-8192-4aeb-8ff6-e799132711ec None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:04.542492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cceb874-8192-4aeb-8ff6-e799132711ec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:04.543829 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cceb874-8192-4aeb-8ff6-e799132711ec None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:04.559994 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 646/1295] 10.222.0.65 () {60 vars in 1141 bytes} [Wed Aug 31 19:03:04 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:06.457812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ab9f3-f1e7-4362-8cdf-d568f87d5c28 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:06.458322 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ab9f3-f1e7-4362-8cdf-d568f87d5c28 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:06.458322 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ab9f3-f1e7-4362-8cdf-d568f87d5c28 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:06.459625 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 650/1296] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:03:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:06.464702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398cc3c-8ada-40ee-8262-ae9c278e3440 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:06.465082 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398cc3c-8ada-40ee-8262-ae9c278e3440 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:06.465270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d398cc3c-8ada-40ee-8262-ae9c278e3440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:06.483404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d398cc3c-8ada-40ee-8262-ae9c278e3440 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:06.484568 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d398cc3c-8ada-40ee-8262-ae9c278e3440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:06.556793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 647/1297] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:06 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 31 19:03:06.819939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ab9f3-f1e7-4362-8cdf-d568f87d5c28 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:06.917985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ab9f3-f1e7-4362-8cdf-d568f87d5c28 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:06.952052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3de262-3752-4bf5-9a59-aea3309b4c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:06.952664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3de262-3752-4bf5-9a59-aea3309b4c59 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:06.952937 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3de262-3752-4bf5-9a59-aea3309b4c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:06.953217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3de262-3752-4bf5-9a59-aea3309b4c59 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:06.953791 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3de262-3752-4bf5-9a59-aea3309b4c59 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:06.955130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3de262-3752-4bf5-9a59-aea3309b4c59 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-2e3de262-3752-4bf5-9a59-aea3309b4c59 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:06.963655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:06.964669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:06.964669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:06.964669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:06.964669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:06.964669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:06.964669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:06.964669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:03:06.964669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:03:06.964669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:03:06.964669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:03:06.964669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:06.964669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:06.964669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:06.964669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:06.964669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:06.964669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:06.965561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:06.965561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:06.965561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:06.965561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:06.965561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:06.965561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:06.965561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:06.965561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:06.965561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:06.965561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:06.965561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:03:06.965561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:03:06.965561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:03:06.965561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:03:06.965561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 651/1298] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 19:03:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:06.970701 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398cc3c-8ada-40ee-8262-ae9c278e3440 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:06.989086 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:07.069704 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d398cc3c-8ada-40ee-8262-ae9c278e3440 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:07.101872 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ce322-a7eb-4a06-a91b-458a10c24ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:07.102406 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ce322-a7eb-4a06-a91b-458a10c24ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:07.102686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ce322-a7eb-4a06-a91b-458a10c24ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:07.102957 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940ce322-a7eb-4a06-a91b-458a10c24ba1 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:07.103579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ce322-a7eb-4a06-a91b-458a10c24ba1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:07.103878 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ce322-a7eb-4a06-a91b-458a10c24ba1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:07.105808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940ce322-a7eb-4a06-a91b-458a10c24ba1 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:07.110752 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 648/1299] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Aug 31 19:03:06 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:07.116695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3de262-3752-4bf5-9a59-aea3309b4c59 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:07.148346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:07.215124 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3de262-3752-4bf5-9a59-aea3309b4c59 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:07.248237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78d51e-6af9-45b5-8c49-0c0b75b36e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:07.249093 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78d51e-6af9-45b5-8c49-0c0b75b36e70 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:07.249396 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78d51e-6af9-45b5-8c49-0c0b75b36e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:07.249667 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78d51e-6af9-45b5-8c49-0c0b75b36e70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:07.250356 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78d51e-6af9-45b5-8c49-0c0b75b36e70 None admin] RBAC: Adding query filter params (interface=public, service_id=ef65c2454aae440f9e039e4d73b85f78, region_id=RegionOne) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:07.250661 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78d51e-6af9-45b5-8c49-0c0b75b36e70 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:07.252005 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78d51e-6af9-45b5-8c49-0c0b75b36e70 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:07.256848 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 652/1300] 10.222.0.65 () {56 vars in 1251 bytes} [Wed Aug 31 19:03:07 2022] GET /identity/v3/endpoints?service_id=ef65c2454aae440f9e039e4d73b85f78&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:07.262431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ce322-a7eb-4a06-a91b-458a10c24ba1 None admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:07.340919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:07.359510 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940ce322-a7eb-4a06-a91b-458a10c24ba1 None admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:07.393163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63e491d-3f3a-4026-9022-95c33133ea34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:07.393777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e491d-3f3a-4026-9022-95c33133ea34 None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:07.394050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e491d-3f3a-4026-9022-95c33133ea34 None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:07.394314 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63e491d-3f3a-4026-9022-95c33133ea34 None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:07.394900 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e491d-3f3a-4026-9022-95c33133ea34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:07.396222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63e491d-3f3a-4026-9022-95c33133ea34 None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:07.402842 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 649/1301] 10.222.0.65 () {56 vars in 1087 bytes} [Wed Aug 31 19:03:07 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:03:09.279701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03e6a7-3032-4a09-9bae-5bb3b808d505 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:09.280217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03e6a7-3032-4a09-9bae-5bb3b808d505 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:09.280217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c03e6a7-3032-4a09-9bae-5bb3b808d505 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:09.281613 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 653/1302] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:03:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:03:09.286674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb0407-6730-4f02-9004-b80ffb64b9b2 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:09.287106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb0407-6730-4f02-9004-b80ffb64b9b2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:09.287202 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb0407-6730-4f02-9004-b80ffb64b9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:09.305508 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c0eb0407-6730-4f02-9004-b80ffb64b9b2 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:03:09.306673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c0eb0407-6730-4f02-9004-b80ffb64b9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:03:09.370385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 650/1303] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:03:09 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 31 19:03:09.645927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03e6a7-3032-4a09-9bae-5bb3b808d505 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:09.739612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:09.747052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c03e6a7-3032-4a09-9bae-5bb3b808d505 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:09.780882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e588e-fcbe-44b2-b378-95f54a15c158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:09.781547 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e588e-fcbe-44b2-b378-95f54a15c158 None admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:09.781842 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e588e-fcbe-44b2-b378-95f54a15c158 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:09.782128 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e588e-fcbe-44b2-b378-95f54a15c158 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:09.782728 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e588e-fcbe-44b2-b378-95f54a15c158 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:09.784130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e588e-fcbe-44b2-b378-95f54a15c158 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application [None req-b69e588e-fcbe-44b2-b378-95f54a15c158 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 31 19:03:09.793680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 31 19:03:09.794820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 31 19:03:09.794820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 31 19:03:09.794820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 31 19:03:09.794820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 31 19:03:09.794820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 31 19:03:09.794820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 31 19:03:09.794820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 31 19:03:09.794820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 31 19:03:09.794820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 31 19:03:09.794820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 31 19:03:09.794820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 31 19:03:09.794820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 31 19:03:09.794820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 31 19:03:09.794820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 31 19:03:09.794820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 31 19:03:09.794820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.get_or_create( Aug 31 19:03:09.796075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 31 19:03:09.796075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application with Lock( Aug 31 19:03:09.796075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 31 19:03:09.796075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self._enter() Aug 31 19:03:09.796075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 31 19:03:09.796075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 31 19:03:09.796075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 31 19:03:09.796075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application return self.creator() Aug 31 19:03:09.796075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 31 19:03:09.796075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application created_value = creator( Aug 31 19:03:09.796075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 31 19:03:09.796075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 31 19:03:09.796075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 31 19:03:09.796075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: ERROR keystone.server.flask.application Aug 31 19:03:09.796075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 654/1304] 10.222.0.65 () {56 vars in 1117 bytes} [Wed Aug 31 19:03:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 31 19:03:09.800693 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb0407-6730-4f02-9004-b80ffb64b9b2 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:09.884080 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:09.908216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eb0407-6730-4f02-9004-b80ffb64b9b2 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:09.942472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b9f3a0-ad6f-4972-9a48-b07b45919abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:09.943112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9f3a0-ad6f-4972-9a48-b07b45919abe None admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:09.943387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9f3a0-ad6f-4972-9a48-b07b45919abe None admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:09.943652 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b9f3a0-ad6f-4972-9a48-b07b45919abe None admin] PATH_INFO: `/v3/services` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:09.944259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9f3a0-ad6f-4972-9a48-b07b45919abe None admin] RBAC: Adding query filter params (name=placement) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:03:09.944558 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9f3a0-ad6f-4972-9a48-b07b45919abe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:09.945913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b9f3a0-ad6f-4972-9a48-b07b45919abe None admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:09.950760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 651/1305] 10.222.0.65 () {56 vars in 1116 bytes} [Wed Aug 31 19:03:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:03:09.956546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e588e-fcbe-44b2-b378-95f54a15c158 None admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:03:10.055570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e588e-fcbe-44b2-b378-95f54a15c158 None admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:03:10.088385 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f04165-e826-4683-afe4-5b02379efab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:03:10.089614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f04165-e826-4683-afe4-5b02379efab3 None admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:03:10.089892 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f04165-e826-4683-afe4-5b02379efab3 None admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:03:10.090162 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f04165-e826-4683-afe4-5b02379efab3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:03:10.090827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f04165-e826-4683-afe4-5b02379efab3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:03:10.092177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f04165-e826-4683-afe4-5b02379efab3 None admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:03:10.110640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 655/1306] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:03:09 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:03:19.948479 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=30 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:03:20.106410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:13.389779 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3aa835-d4de-40e4-8493-f48d775a8264 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:13.390259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3aa835-d4de-40e4-8493-f48d775a8264 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:13.390492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3aa835-d4de-40e4-8493-f48d775a8264 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:13.391917 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 652/1307] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:13.397143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f202f1e-b083-4edd-85ed-b0f69d629ecc None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:13.397573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f202f1e-b083-4edd-85ed-b0f69d629ecc None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:13.397800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f202f1e-b083-4edd-85ed-b0f69d629ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:13.508894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0f202f1e-b083-4edd-85ed-b0f69d629ecc None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:13.510887 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0f202f1e-b083-4edd-85ed-b0f69d629ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:13.654071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 656/1308] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:04:13.939883 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008145d6-4a96-4ed4-a202-6bf5ceba5112 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:13.940246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008145d6-4a96-4ed4-a202-6bf5ceba5112 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:13.940425 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008145d6-4a96-4ed4-a202-6bf5ceba5112 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:13.941817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 653/1309] 10.222.0.65 () {54 vars in 965 bytes} [Wed Aug 31 19:04:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:13.948631 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686b734-c508-4723-ab20-e04d40da587d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:13.948991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686b734-c508-4723-ab20-e04d40da587d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:13.949182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8686b734-c508-4723-ab20-e04d40da587d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:14.038600 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8686b734-c508-4723-ab20-e04d40da587d None None] MFA Rules not processed for user `093edc8c192c469f9532a45bb3326612`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:14.039848 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8686b734-c508-4723-ab20-e04d40da587d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:14.064721 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:14.129647 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 657/1310] 10.222.0.65 () {58 vars in 1062 bytes} [Wed Aug 31 19:04:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:04:14.138941 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008145d6-4a96-4ed4-a202-6bf5ceba5112 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:14.140689 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-008145d6-4a96-4ed4-a202-6bf5ceba5112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:14.266580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008145d6-4a96-4ed4-a202-6bf5ceba5112 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:14.343529 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9e46e-e56c-42c3-a9e8-6f8cd2967348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:14.344059 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9e46e-e56c-42c3-a9e8-6f8cd2967348 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:14.344696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9e46e-e56c-42c3-a9e8-6f8cd2967348 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:14.344696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9e46e-e56c-42c3-a9e8-6f8cd2967348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:14.346428 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-83e9e46e-e56c-42c3-a9e8-6f8cd2967348 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:14.404287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:14.462162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e9e46e-e56c-42c3-a9e8-6f8cd2967348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:14.464027 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e9e46e-e56c-42c3-a9e8-6f8cd2967348 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:14.483838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 654/1311] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:04:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:04:16.683145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996506c6-a79f-4e8c-9633-584cbb52f815 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:16.683677 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996506c6-a79f-4e8c-9633-584cbb52f815 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:16.683677 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996506c6-a79f-4e8c-9633-584cbb52f815 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:16.685090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 658/1312] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:04:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:16.690379 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a1b9e-d91a-471e-a3c0-dbe67a1d6f34 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:16.690841 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a1b9e-d91a-471e-a3c0-dbe67a1d6f34 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:16.690938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727a1b9e-d91a-471e-a3c0-dbe67a1d6f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:16.776430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-727a1b9e-d91a-471e-a3c0-dbe67a1d6f34 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:16.777677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-727a1b9e-d91a-471e-a3c0-dbe67a1d6f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:16.872205 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 655/1313] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:04:17.155706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996506c6-a79f-4e8c-9633-584cbb52f815 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:17.157334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-996506c6-a79f-4e8c-9633-584cbb52f815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:17.260250 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996506c6-a79f-4e8c-9633-584cbb52f815 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:17.330434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:17.333416 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49af68fa-b76d-4d0c-b754-fda0e2a4df38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:17.334033 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af68fa-b76d-4d0c-b754-fda0e2a4df38 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:17.334522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af68fa-b76d-4d0c-b754-fda0e2a4df38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:17.334848 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af68fa-b76d-4d0c-b754-fda0e2a4df38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:17.336861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-49af68fa-b76d-4d0c-b754-fda0e2a4df38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:17.442243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af68fa-b76d-4d0c-b754-fda0e2a4df38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:17.443583 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49af68fa-b76d-4d0c-b754-fda0e2a4df38 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:17.461165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 659/1314] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:04:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:04:19.656353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d564fc-09d1-4a07-a45c-b86630470e21 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:19.656872 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d564fc-09d1-4a07-a45c-b86630470e21 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:19.656872 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d564fc-09d1-4a07-a45c-b86630470e21 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:19.658253 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 656/1315] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:19.663445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bae5b-bc1a-4ba2-bf9b-721ac2afc589 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:19.663863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bae5b-bc1a-4ba2-bf9b-721ac2afc589 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:19.663972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874bae5b-bc1a-4ba2-bf9b-721ac2afc589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:19.731064 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-874bae5b-bc1a-4ba2-bf9b-721ac2afc589 None None] MFA Rules not processed for user `fd866208c70e43639bb105515ea2921e`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:19.732273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-874bae5b-bc1a-4ba2-bf9b-721ac2afc589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:19.783315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:19.826827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 660/1316] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:19 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:04:20.111443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d564fc-09d1-4a07-a45c-b86630470e21 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:20.113012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-62d564fc-09d1-4a07-a45c-b86630470e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:20.123121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:20.217983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d564fc-09d1-4a07-a45c-b86630470e21 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:20.290169 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2a98b7-23b9-4c11-96e8-504d377db530 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:20.290767 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a98b7-23b9-4c11-96e8-504d377db530 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:20.291048 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a98b7-23b9-4c11-96e8-504d377db530 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:20.291320 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2a98b7-23b9-4c11-96e8-504d377db530 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:20.292917 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2a2a98b7-23b9-4c11-96e8-504d377db530 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:20.353779 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:20.400761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a98b7-23b9-4c11-96e8-504d377db530 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:20.402177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2a98b7-23b9-4c11-96e8-504d377db530 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:20.420309 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 657/1317] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:04:20 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:04:22.856663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e40224-b4bd-4d85-a1c5-af61d4a5cfa6 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.857226 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e40224-b4bd-4d85-a1c5-af61d4a5cfa6 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.857226 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e40224-b4bd-4d85-a1c5-af61d4a5cfa6 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.858573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 661/1318] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:04:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:22.863799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56d235-b99c-4d0a-ba8a-a3364d5e2212 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:22.864244 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56d235-b99c-4d0a-ba8a-a3364d5e2212 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:22.864340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c56d235-b99c-4d0a-ba8a-a3364d5e2212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:22.929859 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-3c56d235-b99c-4d0a-ba8a-a3364d5e2212 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:22.931072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3c56d235-b99c-4d0a-ba8a-a3364d5e2212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:23.025018 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 658/1319] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:22 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:04:23.309239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e40224-b4bd-4d85-a1c5-af61d4a5cfa6 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:23.310780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-73e40224-b4bd-4d85-a1c5-af61d4a5cfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:23.413818 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e40224-b4bd-4d85-a1c5-af61d4a5cfa6 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:23.485866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36837d95-ed0b-4768-8d17-3ac9ebdd6c08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:23.486458 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36837d95-ed0b-4768-8d17-3ac9ebdd6c08 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:23.486733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36837d95-ed0b-4768-8d17-3ac9ebdd6c08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:23.487001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36837d95-ed0b-4768-8d17-3ac9ebdd6c08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:23.488583 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-36837d95-ed0b-4768-8d17-3ac9ebdd6c08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:23.495895 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:23.596367 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36837d95-ed0b-4768-8d17-3ac9ebdd6c08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:23.597772 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36837d95-ed0b-4768-8d17-3ac9ebdd6c08 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:23.615334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 662/1320] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:04:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:04:26.190000 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dfc13-75e4-4a1b-9726-d389f56c864b None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.190673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dfc13-75e4-4a1b-9726-d389f56c864b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.190673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4dfc13-75e4-4a1b-9726-d389f56c864b None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.191919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 659/1321] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:26.197816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d8484-2622-4f5b-8812-4badc4355b73 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.198234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d8484-2622-4f5b-8812-4badc4355b73 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.198339 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6d8484-2622-4f5b-8812-4badc4355b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.267916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8d6d8484-2622-4f5b-8812-4badc4355b73 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:26.269106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8d6d8484-2622-4f5b-8812-4badc4355b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:26.329793 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:26.368657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 663/1322] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:04:26.652215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dfc13-75e4-4a1b-9726-d389f56c864b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:26.653856 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1c4dfc13-75e4-4a1b-9726-d389f56c864b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:26.673673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:26.763624 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4dfc13-75e4-4a1b-9726-d389f56c864b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:26.836084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ebeb52-c218-4eea-8165-0cae07301c45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:26.836612 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebeb52-c218-4eea-8165-0cae07301c45 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:26.836891 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebeb52-c218-4eea-8165-0cae07301c45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:26.837164 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ebeb52-c218-4eea-8165-0cae07301c45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:26.838865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e8ebeb52-c218-4eea-8165-0cae07301c45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:26.909937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:26.948399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebeb52-c218-4eea-8165-0cae07301c45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:26.949885 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ebeb52-c218-4eea-8165-0cae07301c45 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:26.968154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 660/1323] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:04:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:04:28.937875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51512150-6d36-45a0-9b57-2d07b8dcd3e8 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:28.938388 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51512150-6d36-45a0-9b57-2d07b8dcd3e8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:28.938388 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51512150-6d36-45a0-9b57-2d07b8dcd3e8 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:28.939852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 664/1324] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:28.944879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0439f-1793-4358-b3d0-a6daca6716bd None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:28.945315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0439f-1793-4358-b3d0-a6daca6716bd None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:28.945564 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0439f-1793-4358-b3d0-a6daca6716bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.011038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-03e0439f-1793-4358-b3d0-a6daca6716bd None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:29.012237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-03e0439f-1793-4358-b3d0-a6daca6716bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:29.106546 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 661/1325] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:04:29.382766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51512150-6d36-45a0-9b57-2d07b8dcd3e8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:29.384319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-51512150-6d36-45a0-9b57-2d07b8dcd3e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:29.503315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51512150-6d36-45a0-9b57-2d07b8dcd3e8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:29.578056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e30d3-9ee2-44a5-ad6d-87549bc53523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:29.578645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e30d3-9ee2-44a5-ad6d-87549bc53523 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:29.578916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e30d3-9ee2-44a5-ad6d-87549bc53523 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:29.579177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e30d3-9ee2-44a5-ad6d-87549bc53523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:29.580742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1b2e30d3-9ee2-44a5-ad6d-87549bc53523 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:29.597174 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:29.688370 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e30d3-9ee2-44a5-ad6d-87549bc53523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:29.689834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e30d3-9ee2-44a5-ad6d-87549bc53523 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:29.707310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 665/1326] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:04:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:04:31.673138 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f4e7b-b5eb-42c9-acf3-b635b1505af8 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:31.673665 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f4e7b-b5eb-42c9-acf3-b635b1505af8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:31.673763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f4e7b-b5eb-42c9-acf3-b635b1505af8 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:31.675109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 662/1327] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:04:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:31.680360 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012da8e-dffa-4612-b57d-e7a53b9549bb None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:31.680816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012da8e-dffa-4612-b57d-e7a53b9549bb None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:31.680922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0012da8e-dffa-4612-b57d-e7a53b9549bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:31.753144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0012da8e-dffa-4612-b57d-e7a53b9549bb None None] MFA Rules not processed for user `fd866208c70e43639bb105515ea2921e`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:31.754387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0012da8e-dffa-4612-b57d-e7a53b9549bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:31.819269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:31.857604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 666/1328] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:31 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:04:32.142141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f4e7b-b5eb-42c9-acf3-b635b1505af8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:32.143670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a59f4e7b-b5eb-42c9-acf3-b635b1505af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:32.180138 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:32.261577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f4e7b-b5eb-42c9-acf3-b635b1505af8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:32.332652 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7be2248-8924-487c-96c5-925101b2295f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:32.333283 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be2248-8924-487c-96c5-925101b2295f service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:32.333592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be2248-8924-487c-96c5-925101b2295f service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:32.333864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be2248-8924-487c-96c5-925101b2295f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:32.335443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b7be2248-8924-487c-96c5-925101b2295f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:32.413210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:32.445213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be2248-8924-487c-96c5-925101b2295f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:32.446586 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7be2248-8924-487c-96c5-925101b2295f service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:32.465017 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 663/1329] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:04:32 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:04:34.608869 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e97f7a-4d14-4b6b-ad1b-787f69734590 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:34.609452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e97f7a-4d14-4b6b-ad1b-787f69734590 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:34.609452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e97f7a-4d14-4b6b-ad1b-787f69734590 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:34.610764 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 667/1330] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:04:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:34.615791 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5148a-5583-4d67-8ef6-5b9ef1e97278 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:34.616197 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5148a-5583-4d67-8ef6-5b9ef1e97278 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:34.616372 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f5148a-5583-4d67-8ef6-5b9ef1e97278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:34.681896 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-54f5148a-5583-4d67-8ef6-5b9ef1e97278 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:34.683091 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-54f5148a-5583-4d67-8ef6-5b9ef1e97278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:34.779537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 664/1331] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:04:35.058704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e97f7a-4d14-4b6b-ad1b-787f69734590 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:35.060244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f3e97f7a-4d14-4b6b-ad1b-787f69734590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:35.162824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e97f7a-4d14-4b6b-ad1b-787f69734590 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:35.234609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132a208-84e2-466c-93c1-a93e968bdbff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:35.235212 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132a208-84e2-466c-93c1-a93e968bdbff service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:35.235482 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132a208-84e2-466c-93c1-a93e968bdbff service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:35.235747 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132a208-84e2-466c-93c1-a93e968bdbff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:35.237359 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6132a208-84e2-466c-93c1-a93e968bdbff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:35.263628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:35.344225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132a208-84e2-466c-93c1-a93e968bdbff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:35.345689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132a208-84e2-466c-93c1-a93e968bdbff service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:35.363697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 668/1332] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:04:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:04:37.322215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30d837-27d5-4154-9047-c8209e9f75e7 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:37.322766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30d837-27d5-4154-9047-c8209e9f75e7 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:37.322766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30d837-27d5-4154-9047-c8209e9f75e7 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:37.324092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 665/1333] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:37.329157 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c7d4-e582-42f1-9381-a96f4882fa73 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:37.329598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c7d4-e582-42f1-9381-a96f4882fa73 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:37.329712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c7d4-e582-42f1-9381-a96f4882fa73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:37.394331 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-88a2c7d4-e582-42f1-9381-a96f4882fa73 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:37.395511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-88a2c7d4-e582-42f1-9381-a96f4882fa73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:37.468782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:37.492717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 669/1334] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:04:37.788669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bd0ce-32a7-4600-b4c6-050736b9babb None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:37.788949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bd0ce-32a7-4600-b4c6-050736b9babb None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:37.789264 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bd0ce-32a7-4600-b4c6-050736b9babb None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:37.791162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 666/1335] 10.222.0.65 () {54 vars in 965 bytes} [Wed Aug 31 19:04:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:37.797112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fd1c0-0b11-4754-a313-42be000ed31e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:37.797490 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fd1c0-0b11-4754-a313-42be000ed31e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:37.797773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8fd1c0-0b11-4754-a313-42be000ed31e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:37.875618 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6d8fd1c0-0b11-4754-a313-42be000ed31e None None] MFA Rules not processed for user `093edc8c192c469f9532a45bb3326612`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:37.876826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6d8fd1c0-0b11-4754-a313-42be000ed31e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:37.950561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 670/1336] 10.222.0.65 () {58 vars in 1062 bytes} [Wed Aug 31 19:04:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:04:37.959506 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bd0ce-32a7-4600-b4c6-050736b9babb None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:37.960998 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d46bd0ce-32a7-4600-b4c6-050736b9babb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:38.000882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:38.063573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bd0ce-32a7-4600-b4c6-050736b9babb None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:38.133372 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80508957-2097-4c76-9c10-c4fc8203d710 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:38.134030 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80508957-2097-4c76-9c10-c4fc8203d710 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:38.134291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80508957-2097-4c76-9c10-c4fc8203d710 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:38.134670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80508957-2097-4c76-9c10-c4fc8203d710 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:38.136743 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-80508957-2097-4c76-9c10-c4fc8203d710 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:38.227752 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:38.253494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80508957-2097-4c76-9c10-c4fc8203d710 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:38.254912 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80508957-2097-4c76-9c10-c4fc8203d710 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:38.272081 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 667/1337] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:04:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:04:41.000264 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef25a6e-7170-41c6-992d-e7324a2b5544 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.000801 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef25a6e-7170-41c6-992d-e7324a2b5544 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.000801 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef25a6e-7170-41c6-992d-e7324a2b5544 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.002173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 671/1338] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:41.007359 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd15ad5-5066-42f5-98a0-dfbde176b6bc None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.007772 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd15ad5-5066-42f5-98a0-dfbde176b6bc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.007909 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd15ad5-5066-42f5-98a0-dfbde176b6bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.072771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4fd15ad5-5066-42f5-98a0-dfbde176b6bc None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:41.073993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4fd15ad5-5066-42f5-98a0-dfbde176b6bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:41.168628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 668/1339] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:04:41.453992 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef25a6e-7170-41c6-992d-e7324a2b5544 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:41.455518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cef25a6e-7170-41c6-992d-e7324a2b5544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:41.537844 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:41.566064 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef25a6e-7170-41c6-992d-e7324a2b5544 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:41.638326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9ce16-3259-4451-a611-bf071ddf832e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:41.638935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9ce16-3259-4451-a611-bf071ddf832e service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:41.639200 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9ce16-3259-4451-a611-bf071ddf832e service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:41.639469 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9ce16-3259-4451-a611-bf071ddf832e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:41.641129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9cb9ce16-3259-4451-a611-bf071ddf832e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:41.746503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9ce16-3259-4451-a611-bf071ddf832e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:41.748097 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9ce16-3259-4451-a611-bf071ddf832e service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:41.766127 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 672/1340] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:04:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:04:45.724685 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384ee13-b855-4659-974f-84cc17556bad None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:45.725217 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384ee13-b855-4659-974f-84cc17556bad None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:45.725331 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c384ee13-b855-4659-974f-84cc17556bad None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:45.726587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 669/1341] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:45.732302 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f18f5-0762-48ea-adaa-ea305bdf5b47 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:45.732729 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f18f5-0762-48ea-adaa-ea305bdf5b47 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:45.732840 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1f18f5-0762-48ea-adaa-ea305bdf5b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:45.738349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:45.800904 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6d1f18f5-0762-48ea-adaa-ea305bdf5b47 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:45.802130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6d1f18f5-0762-48ea-adaa-ea305bdf5b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:45.897118 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 673/1342] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:04:46.182616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384ee13-b855-4659-974f-84cc17556bad None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:46.184151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c384ee13-b855-4659-974f-84cc17556bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:46.233725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:46.289878 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c384ee13-b855-4659-974f-84cc17556bad None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:46.365237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fef7c-9381-4297-be01-c87e90c8ab22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:46.365843 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fef7c-9381-4297-be01-c87e90c8ab22 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:46.366123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fef7c-9381-4297-be01-c87e90c8ab22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:46.366394 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fef7c-9381-4297-be01-c87e90c8ab22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:46.367984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3c5fef7c-9381-4297-be01-c87e90c8ab22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:46.463029 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:46.481413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fef7c-9381-4297-be01-c87e90c8ab22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:46.482863 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5fef7c-9381-4297-be01-c87e90c8ab22 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:46.501601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 670/1343] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:04:49.027180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9ab7f-485e-4505-8d5e-2b64bb87f551 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.027762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9ab7f-485e-4505-8d5e-2b64bb87f551 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.027762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c9ab7f-485e-4505-8d5e-2b64bb87f551 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.029113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 674/1344] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:04:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:49.034949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d428b9c-12f9-48dc-891b-58b695714a6d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.035482 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d428b9c-12f9-48dc-891b-58b695714a6d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.035482 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d428b9c-12f9-48dc-891b-58b695714a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.106670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4d428b9c-12f9-48dc-891b-58b695714a6d None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:49.107914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4d428b9c-12f9-48dc-891b-58b695714a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:49.205848 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 671/1345] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:04:49.491253 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9ab7f-485e-4505-8d5e-2b64bb87f551 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:49.492809 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-01c9ab7f-485e-4505-8d5e-2b64bb87f551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:49.585134 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:49.606644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c9ab7f-485e-4505-8d5e-2b64bb87f551 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:49.687203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bde57ed-c31c-4dbd-b17d-4cca4722cd2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:49.687907 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde57ed-c31c-4dbd-b17d-4cca4722cd2d service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:49.688248 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde57ed-c31c-4dbd-b17d-4cca4722cd2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:49.688582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bde57ed-c31c-4dbd-b17d-4cca4722cd2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:49.690286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6bde57ed-c31c-4dbd-b17d-4cca4722cd2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:49.803824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde57ed-c31c-4dbd-b17d-4cca4722cd2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:49.805288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bde57ed-c31c-4dbd-b17d-4cca4722cd2d service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:49.824236 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 675/1346] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:04:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:04:53.671242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34346fd-c886-48dc-ad4a-a8e5af89f9bf None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.671750 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34346fd-c886-48dc-ad4a-a8e5af89f9bf None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.671849 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34346fd-c886-48dc-ad4a-a8e5af89f9bf None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.673174 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 672/1347] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:04:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:53.678509 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d4e53-9abd-4e93-889b-5969aad32086 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:53.679020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d4e53-9abd-4e93-889b-5969aad32086 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:53.679104 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0d4e53-9abd-4e93-889b-5969aad32086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:53.695630 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:53.773101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-1e0d4e53-9abd-4e93-889b-5969aad32086 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:53.774348 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1e0d4e53-9abd-4e93-889b-5969aad32086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:53.883494 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 676/1348] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:04:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:04:54.161836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34346fd-c886-48dc-ad4a-a8e5af89f9bf None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:04:54.163374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b34346fd-c886-48dc-ad4a-a8e5af89f9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:54.229824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:54.277811 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34346fd-c886-48dc-ad4a-a8e5af89f9bf None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:04:54.354181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22657407-cf34-4107-8f8c-f1ca1bde28b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:04:54.354788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22657407-cf34-4107-8f8c-f1ca1bde28b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:54.355064 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22657407-cf34-4107-8f8c-f1ca1bde28b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:54.355332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22657407-cf34-4107-8f8c-f1ca1bde28b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:54.356940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-22657407-cf34-4107-8f8c-f1ca1bde28b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:54.465425 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:04:54.469584 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22657407-cf34-4107-8f8c-f1ca1bde28b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:04:54.471031 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22657407-cf34-4107-8f8c-f1ca1bde28b9 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:04:54.489723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 673/1349] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:04:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:04:59.415830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b674d00-5387-43b0-8893-b06389303fb3 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.416338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b674d00-5387-43b0-8893-b06389303fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.416446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b674d00-5387-43b0-8893-b06389303fb3 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.418890 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 677/1350] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 19:04:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:04:59.424614 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46c339-71d8-4442-9f94-f2079502ea22 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:04:59.425588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46c339-71d8-4442-9f94-f2079502ea22 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:04:59.426015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46c339-71d8-4442-9f94-f2079502ea22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:04:59.530148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-6f46c339-71d8-4442-9f94-f2079502ea22 None None] MFA Rules not processed for user `74f1846e395e4d5483fcbb41eb28a77d`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:04:59.531376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6f46c339-71d8-4442-9f94-f2079502ea22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:04:59.695617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 674/1351] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 19:04:59 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:03.866916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bd664-ed74-47a3-9541-b8e4c115798c None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:03.867507 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bd664-ed74-47a3-9541-b8e4c115798c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:03.867507 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0bd664-ed74-47a3-9541-b8e4c115798c None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:03.869581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 678/1352] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 19:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:03.874849 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d017f-c477-4151-8c1f-1cd8ea9d6f89 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:03.875346 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d017f-c477-4151-8c1f-1cd8ea9d6f89 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:03.875455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d017f-c477-4151-8c1f-1cd8ea9d6f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:03.889220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=59 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:03.934816 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:03.960219 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-654d017f-c477-4151-8c1f-1cd8ea9d6f89 None None] MFA Rules not processed for user `74f1846e395e4d5483fcbb41eb28a77d`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:03.961761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-654d017f-c477-4151-8c1f-1cd8ea9d6f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:04.074943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 675/1353] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 19:05:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:05.544667 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebb8ab-7f04-4d00-9f14-cdcc13855330 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:05.545219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebb8ab-7f04-4d00-9f14-cdcc13855330 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:05.545308 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ebb8ab-7f04-4d00-9f14-cdcc13855330 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:05.546664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 679/1354] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 19:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:05.551693 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27da7c-af7d-4f23-9ef4-bbbe9864971d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:05.552125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27da7c-af7d-4f23-9ef4-bbbe9864971d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:05.552251 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c27da7c-af7d-4f23-9ef4-bbbe9864971d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:05.630934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0c27da7c-af7d-4f23-9ef4-bbbe9864971d None None] MFA Rules not processed for user `74f1846e395e4d5483fcbb41eb28a77d`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:05.632133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0c27da7c-af7d-4f23-9ef4-bbbe9864971d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:05.674194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:05.738475 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 676/1355] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 19:05:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:05:09.045327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8c821-2cbd-41c3-b466-280b964ea28e None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:09.045826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8c821-2cbd-41c3-b466-280b964ea28e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:09.046038 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a8c821-2cbd-41c3-b466-280b964ea28e None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:09.048268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 680/1356] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:05:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:09.053752 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca07e3-2e23-423e-85de-00dfef9b2ae5 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:09.054212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca07e3-2e23-423e-85de-00dfef9b2ae5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:09.054409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbca07e3-2e23-423e-85de-00dfef9b2ae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:09.150921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-cbca07e3-2e23-423e-85de-00dfef9b2ae5 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:09.152485 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cbca07e3-2e23-423e-85de-00dfef9b2ae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:09.286680 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 677/1357] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:09.588421 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54868d5-a142-4524-a358-39c64193926d None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:09.588708 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54868d5-a142-4524-a358-39c64193926d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:09.588980 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54868d5-a142-4524-a358-39c64193926d None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:09.590385 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 681/1358] 10.222.0.65 () {54 vars in 962 bytes} [Wed Aug 31 19:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:09.598400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bf80c-cc22-4679-992a-067fde2c49a4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:09.598761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bf80c-cc22-4679-992a-067fde2c49a4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:09.599278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4bf80c-cc22-4679-992a-067fde2c49a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:09.621380 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:09.691846 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ae4bf80c-cc22-4679-992a-067fde2c49a4 None None] MFA Rules not processed for user `673a72ac9a9847e3812007b66ac7a5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:09.693117 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ae4bf80c-cc22-4679-992a-067fde2c49a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:09.823202 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 678/1359] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Aug 31 19:05:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:05:09.830863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54868d5-a142-4524-a358-39c64193926d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:09.832405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f54868d5-a142-4524-a358-39c64193926d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:09.995790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54868d5-a142-4524-a358-39c64193926d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:10.077678 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:10.122957 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58927e6a-bcd6-42c5-8af8-2093800fc8ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:10.124075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58927e6a-bcd6-42c5-8af8-2093800fc8ee service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:10.124452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58927e6a-bcd6-42c5-8af8-2093800fc8ee service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:10.124966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58927e6a-bcd6-42c5-8af8-2093800fc8ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:10.126926 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-58927e6a-bcd6-42c5-8af8-2093800fc8ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:10.230328 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58927e6a-bcd6-42c5-8af8-2093800fc8ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:10.233620 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58927e6a-bcd6-42c5-8af8-2093800fc8ee service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:10.257486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 682/1360] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:05:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:05:12.423063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2d9fe-dead-4df4-b47f-3e168c92f4cc None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:12.423645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2d9fe-dead-4df4-b47f-3e168c92f4cc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:12.423645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b2d9fe-dead-4df4-b47f-3e168c92f4cc None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:12.428790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 679/1361] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:05:12 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:12.430379 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbb760-7739-4791-a70f-c35bc60292e7 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:12.430819 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbb760-7739-4791-a70f-c35bc60292e7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:12.430936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbb760-7739-4791-a70f-c35bc60292e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:12.505054 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-52bbb760-7739-4791-a70f-c35bc60292e7 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:12.506843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-52bbb760-7739-4791-a70f-c35bc60292e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:12.556456 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:12.633803 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 683/1362] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:05:12.927877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2d9fe-dead-4df4-b47f-3e168c92f4cc None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:12.929361 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-46b2d9fe-dead-4df4-b47f-3e168c92f4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:12.998688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:13.090346 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b2d9fe-dead-4df4-b47f-3e168c92f4cc None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:13.229037 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eb2a18-18e8-413f-baf2-831ca9f26001 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:13.229880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb2a18-18e8-413f-baf2-831ca9f26001 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:13.230286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb2a18-18e8-413f-baf2-831ca9f26001 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:13.230619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eb2a18-18e8-413f-baf2-831ca9f26001 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:13.232387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e7eb2a18-18e8-413f-baf2-831ca9f26001 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:13.257291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:13.329844 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb2a18-18e8-413f-baf2-831ca9f26001 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:13.331443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7eb2a18-18e8-413f-baf2-831ca9f26001 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:13.350548 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 680/1363] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:05:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:05:15.599992 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f15ace-76aa-4fa0-99c5-5070f76235b3 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:15.600550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f15ace-76aa-4fa0-99c5-5070f76235b3 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:15.600815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f15ace-76aa-4fa0-99c5-5070f76235b3 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:15.602597 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 684/1364] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:05:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:15.607613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb30c49-22b7-417e-95e0-746c6ed51388 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:15.608077 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb30c49-22b7-417e-95e0-746c6ed51388 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:15.608156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb30c49-22b7-417e-95e0-746c6ed51388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:15.675200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0cb30c49-22b7-417e-95e0-746c6ed51388 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:15.676540 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0cb30c49-22b7-417e-95e0-746c6ed51388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:15.773397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:15.776191 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 681/1365] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:16.073373 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2556bd8-b55d-455b-b2bd-4a62d3300fca None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:16.073777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2556bd8-b55d-455b-b2bd-4a62d3300fca None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:16.074159 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2556bd8-b55d-455b-b2bd-4a62d3300fca None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:16.075776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 685/1366] 10.222.0.65 () {54 vars in 962 bytes} [Wed Aug 31 19:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:16.081232 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad949f14-bc48-4ea1-bd5e-a4bce28f853b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:16.081686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad949f14-bc48-4ea1-bd5e-a4bce28f853b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:16.081987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad949f14-bc48-4ea1-bd5e-a4bce28f853b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:16.161505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ad949f14-bc48-4ea1-bd5e-a4bce28f853b None None] MFA Rules not processed for user `673a72ac9a9847e3812007b66ac7a5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:16.162722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ad949f14-bc48-4ea1-bd5e-a4bce28f853b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:16.284325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 682/1367] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Aug 31 19:05:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:16.293330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2556bd8-b55d-455b-b2bd-4a62d3300fca None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:16.295003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d2556bd8-b55d-455b-b2bd-4a62d3300fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:16.453261 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2556bd8-b55d-455b-b2bd-4a62d3300fca None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:16.475611 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:16.575051 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c093c02-2fc8-48a4-bafd-0383300f4c4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:16.575723 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c093c02-2fc8-48a4-bafd-0383300f4c4e service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:16.576102 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c093c02-2fc8-48a4-bafd-0383300f4c4e service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:16.576492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c093c02-2fc8-48a4-bafd-0383300f4c4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:16.578213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1c093c02-2fc8-48a4-bafd-0383300f4c4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:16.675529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c093c02-2fc8-48a4-bafd-0383300f4c4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:16.677315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c093c02-2fc8-48a4-bafd-0383300f4c4e service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:16.697916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 686/1368] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:05:17.242248 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b59eb-4e75-45a0-b36c-e6286169b7df None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.243310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b59eb-4e75-45a0-b36c-e6286169b7df None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.243310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4b59eb-4e75-45a0-b36c-e6286169b7df None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.244848 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 683/1369] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:17.250151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2186c-345a-4611-ad09-06b7f32b1a7a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:17.250429 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2186c-345a-4611-ad09-06b7f32b1a7a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:17.250696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2186c-345a-4611-ad09-06b7f32b1a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:17.284055 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:17.322388 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-d4e2186c-345a-4611-ad09-06b7f32b1a7a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:17.323592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d4e2186c-345a-4611-ad09-06b7f32b1a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:17.433236 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 687/1370] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:18.018408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6a3e1-2eff-4520-b89c-70c368817fc3 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.018408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6a3e1-2eff-4520-b89c-70c368817fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.018408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb6a3e1-2eff-4520-b89c-70c368817fc3 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.019163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 684/1371] 10.222.0.65 () {54 vars in 964 bytes} [Wed Aug 31 19:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:18.024438 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99110dd8-88ec-4da8-92b2-536ab842393a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.024881 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99110dd8-88ec-4da8-92b2-536ab842393a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.024985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99110dd8-88ec-4da8-92b2-536ab842393a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.106292 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-99110dd8-88ec-4da8-92b2-536ab842393a None None] MFA Rules not processed for user `5d157abcf94745a5b92d6df64d872f73`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:18.107500 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-99110dd8-88ec-4da8-92b2-536ab842393a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:18.156408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:18.193382 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 688/1372] 10.222.0.65 () {58 vars in 1061 bytes} [Wed Aug 31 19:05:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:18.200873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6a3e1-2eff-4520-b89c-70c368817fc3 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:18.202430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0eb6a3e1-2eff-4520-b89c-70c368817fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:18.255296 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:18.312768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb6a3e1-2eff-4520-b89c-70c368817fc3 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:18.393027 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddb4e1-d022-4f9f-aef3-f58710a111ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:18.393962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddb4e1-d022-4f9f-aef3-f58710a111ea service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.394558 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddb4e1-d022-4f9f-aef3-f58710a111ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.394886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddb4e1-d022-4f9f-aef3-f58710a111ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.396532 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9eddb4e1-d022-4f9f-aef3-f58710a111ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:18.519497 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:18.537885 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddb4e1-d022-4f9f-aef3-f58710a111ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:18.539364 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddb4e1-d022-4f9f-aef3-f58710a111ea service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:18.559807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 685/1373] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:05:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:05:18.676994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57c1cc-84c2-4fd4-975c-20a78dd58f60 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.677667 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57c1cc-84c2-4fd4-975c-20a78dd58f60 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.677970 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa57c1cc-84c2-4fd4-975c-20a78dd58f60 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.679636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 689/1374] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:18.685457 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ebfb6-a2f3-4908-8e84-e68d4b7660da None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:18.685744 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ebfb6-a2f3-4908-8e84-e68d4b7660da None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:18.686017 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443ebfb6-a2f3-4908-8e84-e68d4b7660da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:18.773732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-443ebfb6-a2f3-4908-8e84-e68d4b7660da None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:18.775009 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-443ebfb6-a2f3-4908-8e84-e68d4b7660da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:18.882419 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 686/1375] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:19.168280 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57c1cc-84c2-4fd4-975c-20a78dd58f60 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:19.169917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-aa57c1cc-84c2-4fd4-975c-20a78dd58f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:19.322174 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa57c1cc-84c2-4fd4-975c-20a78dd58f60 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:19.386605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:19.451184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2552ea7-e75b-499e-aa3a-94d821402eba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:19.451710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2552ea7-e75b-499e-aa3a-94d821402eba service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:19.451988 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2552ea7-e75b-499e-aa3a-94d821402eba service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:19.452261 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2552ea7-e75b-499e-aa3a-94d821402eba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:19.453872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c2552ea7-e75b-499e-aa3a-94d821402eba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:19.544032 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2552ea7-e75b-499e-aa3a-94d821402eba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:19.545648 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2552ea7-e75b-499e-aa3a-94d821402eba service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:19.563672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 690/1376] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:05:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:05:21.497355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100c30d-9082-48e6-a14d-6a250bb6cd98 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.497856 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100c30d-9082-48e6-a14d-6a250bb6cd98 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.497968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100c30d-9082-48e6-a14d-6a250bb6cd98 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.499233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 687/1377] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:21.504755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc891e8a-69d1-41ab-be85-81557b473c8f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:21.505177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc891e8a-69d1-41ab-be85-81557b473c8f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:21.505300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc891e8a-69d1-41ab-be85-81557b473c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:21.591970 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-bc891e8a-69d1-41ab-be85-81557b473c8f None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:21.593230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bc891e8a-69d1-41ab-be85-81557b473c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:21.603716 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:21.697205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 691/1378] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:21.986152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100c30d-9082-48e6-a14d-6a250bb6cd98 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:21.987663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b100c30d-9082-48e6-a14d-6a250bb6cd98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:22.075535 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:22.180537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100c30d-9082-48e6-a14d-6a250bb6cd98 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:22.311416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414e54c8-2840-4d5e-a839-8df45ed46878 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:22.312023 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e54c8-2840-4d5e-a839-8df45ed46878 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:22.312303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e54c8-2840-4d5e-a839-8df45ed46878 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:22.312575 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414e54c8-2840-4d5e-a839-8df45ed46878 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:22.314192 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-414e54c8-2840-4d5e-a839-8df45ed46878 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:22.335096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:22.412497 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e54c8-2840-4d5e-a839-8df45ed46878 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:22.414133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414e54c8-2840-4d5e-a839-8df45ed46878 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:22.434130 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 688/1379] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:05:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:05:24.382605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d9a26-501f-4ab2-8cbc-e4aec460e116 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.383137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d9a26-501f-4ab2-8cbc-e4aec460e116 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.383296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d9a26-501f-4ab2-8cbc-e4aec460e116 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.384803 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 692/1380] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:24.389840 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5202192-b2f6-4786-bb85-6cae5d0f1ed8 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:24.390349 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5202192-b2f6-4786-bb85-6cae5d0f1ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:24.390502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5202192-b2f6-4786-bb85-6cae5d0f1ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:24.460437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b5202192-b2f6-4786-bb85-6cae5d0f1ed8 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:24.462105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b5202192-b2f6-4786-bb85-6cae5d0f1ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:24.559414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:24.566699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 689/1381] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:24.860778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d9a26-501f-4ab2-8cbc-e4aec460e116 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:24.862633 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-175d9a26-501f-4ab2-8cbc-e4aec460e116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:25.019997 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:25.033581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d9a26-501f-4ab2-8cbc-e4aec460e116 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:25.163846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e3113b-41cc-4320-926c-ce9154a1a382 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:25.164660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3113b-41cc-4320-926c-ce9154a1a382 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:25.165140 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3113b-41cc-4320-926c-ce9154a1a382 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:25.165550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e3113b-41cc-4320-926c-ce9154a1a382 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:25.167293 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e9e3113b-41cc-4320-926c-ce9154a1a382 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:25.271920 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3113b-41cc-4320-926c-ce9154a1a382 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:25.273568 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e3113b-41cc-4320-926c-ce9154a1a382 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:25.285512 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:25.295846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 693/1382] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:05:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:05:27.294976 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30931a0c-6d9f-4122-af89-a7217ddd25e9 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.295558 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30931a0c-6d9f-4122-af89-a7217ddd25e9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.295558 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30931a0c-6d9f-4122-af89-a7217ddd25e9 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.296914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 690/1383] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:27.302272 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33f43e-a56e-465d-8c7f-570d903d01cf None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:27.302691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33f43e-a56e-465d-8c7f-570d903d01cf None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:27.302892 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f33f43e-a56e-465d-8c7f-570d903d01cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:27.376061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4f33f43e-a56e-465d-8c7f-570d903d01cf None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:27.377707 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4f33f43e-a56e-465d-8c7f-570d903d01cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:27.488719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 694/1384] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:27.775659 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30931a0c-6d9f-4122-af89-a7217ddd25e9 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:27.777224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-30931a0c-6d9f-4122-af89-a7217ddd25e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:27.922313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30931a0c-6d9f-4122-af89-a7217ddd25e9 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:27.992984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:28.047023 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67736f69-ccdc-46b3-9559-44ebbdfebb82 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:28.047546 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67736f69-ccdc-46b3-9559-44ebbdfebb82 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:28.047818 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67736f69-ccdc-46b3-9559-44ebbdfebb82 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:28.048080 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67736f69-ccdc-46b3-9559-44ebbdfebb82 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:28.049688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-67736f69-ccdc-46b3-9559-44ebbdfebb82 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:28.141175 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67736f69-ccdc-46b3-9559-44ebbdfebb82 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:28.142792 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67736f69-ccdc-46b3-9559-44ebbdfebb82 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:28.168268 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 691/1385] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:05:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:05:30.062758 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339994f-f30d-4594-89e0-1bffba6f04cc None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.063431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339994f-f30d-4594-89e0-1bffba6f04cc None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.063431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b339994f-f30d-4594-89e0-1bffba6f04cc None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.064640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 695/1386] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:30.069773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149d428-952e-46a8-9789-70cc93bf2ec5 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.070329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149d428-952e-46a8-9789-70cc93bf2ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.070424 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5149d428-952e-46a8-9789-70cc93bf2ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.146642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-5149d428-952e-46a8-9789-70cc93bf2ec5 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:30.147833 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5149d428-952e-46a8-9789-70cc93bf2ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:30.181488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:30.266606 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 692/1387] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:30.556880 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339994f-f30d-4594-89e0-1bffba6f04cc None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:30.558467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b339994f-f30d-4594-89e0-1bffba6f04cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:30.639235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:30.712400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b339994f-f30d-4594-89e0-1bffba6f04cc None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:30.829465 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659eea94-3109-475e-8c53-750bd9562848 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:30.829994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659eea94-3109-475e-8c53-750bd9562848 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:30.830266 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659eea94-3109-475e-8c53-750bd9562848 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:30.830531 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659eea94-3109-475e-8c53-750bd9562848 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:30.832127 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-659eea94-3109-475e-8c53-750bd9562848 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:30.873202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:30.935763 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659eea94-3109-475e-8c53-750bd9562848 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:30.937408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659eea94-3109-475e-8c53-750bd9562848 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:30.955806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 696/1388] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:05:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:05:32.900432 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb32a82-9190-444f-ac75-3541f76936fc None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:32.901084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb32a82-9190-444f-ac75-3541f76936fc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:32.901084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb32a82-9190-444f-ac75-3541f76936fc None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:32.902358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 693/1389] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:32.907440 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028d2a9-5890-463b-86e8-844ee41f2f30 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:32.907866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028d2a9-5890-463b-86e8-844ee41f2f30 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:32.908046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a028d2a9-5890-463b-86e8-844ee41f2f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:32.973700 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a028d2a9-5890-463b-86e8-844ee41f2f30 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:32.974896 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a028d2a9-5890-463b-86e8-844ee41f2f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:33.071770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 697/1390] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:33.359255 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb32a82-9190-444f-ac75-3541f76936fc None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:33.360803 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-fbb32a82-9190-444f-ac75-3541f76936fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:33.505510 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb32a82-9190-444f-ac75-3541f76936fc None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:33.522085 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:33.619716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc6d92-20eb-40d6-bdda-3486127bdee0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:33.620237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc6d92-20eb-40d6-bdda-3486127bdee0 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:33.620533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc6d92-20eb-40d6-bdda-3486127bdee0 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:33.620793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc6d92-20eb-40d6-bdda-3486127bdee0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:33.622761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-45bc6d92-20eb-40d6-bdda-3486127bdee0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:33.698359 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee897f7-8676-4b69-98a0-07d619aed325 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:33.698636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee897f7-8676-4b69-98a0-07d619aed325 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:33.698913 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee897f7-8676-4b69-98a0-07d619aed325 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:33.700396 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 698/1391] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:05:33.704996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe5f34-9230-4a84-8e0c-67d31d9fb395 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:33.705301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe5f34-9230-4a84-8e0c-67d31d9fb395 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:33.705573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe5f34-9230-4a84-8e0c-67d31d9fb395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:33.713005 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc6d92-20eb-40d6-bdda-3486127bdee0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:33.714678 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc6d92-20eb-40d6-bdda-3486127bdee0 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:33.721394 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:33.733552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 694/1392] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:05:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:05:33.774830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f2fe5f34-9230-4a84-8e0c-67d31d9fb395 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:33.776073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f2fe5f34-9230-4a84-8e0c-67d31d9fb395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:33.884570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 699/1393] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:05:34.188708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3a99c-9def-44f0-a9d4-62fc3d40b1c0 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:34.189210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3a99c-9def-44f0-a9d4-62fc3d40b1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:34.189619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e3a99c-9def-44f0-a9d4-62fc3d40b1c0 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:34.191907 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 695/1394] 10.222.0.65 () {54 vars in 967 bytes} [Wed Aug 31 19:05:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:34.199414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dce64-e333-41e2-88a0-abcba8dfa603 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:34.199876 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dce64-e333-41e2-88a0-abcba8dfa603 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:34.199981 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008dce64-e333-41e2-88a0-abcba8dfa603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:34.281771 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-008dce64-e333-41e2-88a0-abcba8dfa603 None None] MFA Rules not processed for user `1863336fecad4987acdfd0b3f7b2f70f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:34.282984 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-008dce64-e333-41e2-88a0-abcba8dfa603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:34.317223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:34.359245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 700/1395] 10.222.0.65 () {58 vars in 1064 bytes} [Wed Aug 31 19:05:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:34.366832 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3a99c-9def-44f0-a9d4-62fc3d40b1c0 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:34.368855 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d2e3a99c-9def-44f0-a9d4-62fc3d40b1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:34.392194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:34.472258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e3a99c-9def-44f0-a9d4-62fc3d40b1c0 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:34.545649 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46735b4f-d359-48b6-8458-056709da255f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1863336fecad4987acdfd0b3f7b2f70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:34.546259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735b4f-d359-48b6-8458-056709da255f service glance] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:34.546537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735b4f-d359-48b6-8458-056709da255f service glance] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:34.546804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46735b4f-d359-48b6-8458-056709da255f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:34.548395 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-46735b4f-d359-48b6-8458-056709da255f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:34.624361 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:34.660256 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46735b4f-d359-48b6-8458-056709da255f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:34.661719 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46735b4f-d359-48b6-8458-056709da255f service glance] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:34.679766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 696/1396] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:05:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:05:35.415240 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23768a13-4f53-44b7-a56b-f7f70badb606 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.415705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23768a13-4f53-44b7-a56b-f7f70badb606 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.415806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23768a13-4f53-44b7-a56b-f7f70badb606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.482040 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-23768a13-4f53-44b7-a56b-f7f70badb606 None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:35.483254 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-23768a13-4f53-44b7-a56b-f7f70badb606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:35.570186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 701/1397] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 19:05:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:35.615686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312f2cb-f74c-4200-bbd8-5dba7199051b None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.616203 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312f2cb-f74c-4200-bbd8-5dba7199051b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.616373 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312f2cb-f74c-4200-bbd8-5dba7199051b None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.618372 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 697/1398] 10.222.0.65 () {54 vars in 943 bytes} [Wed Aug 31 19:05:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:35.625002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081d122-be17-402e-8204-0cc5eb55ef09 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.625314 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081d122-be17-402e-8204-0cc5eb55ef09 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.625589 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7081d122-be17-402e-8204-0cc5eb55ef09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.683716 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:35.702853 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-7081d122-be17-402e-8204-0cc5eb55ef09 None None] MFA Rules not processed for user `586715d11b924e2cb6f44030efd77619`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:35.704396 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7081d122-be17-402e-8204-0cc5eb55ef09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:35.704565 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94bdaf-96d1-4e5a-81d0-d10a7357194a None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.704857 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94bdaf-96d1-4e5a-81d0-d10a7357194a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.705133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd94bdaf-96d1-4e5a-81d0-d10a7357194a None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.706510 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 698/1399] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:05:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:35.711277 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b47c14-85e7-433e-b1c5-dfbf6f144d0c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.711565 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b47c14-85e7-433e-b1c5-dfbf6f144d0c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.711846 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b47c14-85e7-433e-b1c5-dfbf6f144d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.777999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c1b47c14-85e7-433e-b1c5-dfbf6f144d0c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:35.779217 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c1b47c14-85e7-433e-b1c5-dfbf6f144d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:35.812320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 702/1400] 10.222.0.65 () {58 vars in 1040 bytes} [Wed Aug 31 19:05:35 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:35.822587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081d122-be17-402e-8204-0cc5eb55ef09 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:35.832298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7081d122-be17-402e-8204-0cc5eb55ef09 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:35.878125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 699/1401] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:05:35.932551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:35.938304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d2460-b6f7-4ed7-ae10-d2fefecd07e1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:35.938906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d2460-b6f7-4ed7-ae10-d2fefecd07e1 service swift] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:35.939183 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d2460-b6f7-4ed7-ae10-d2fefecd07e1 service swift] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:35.939455 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d2460-b6f7-4ed7-ae10-d2fefecd07e1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:35.941089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0e8d2460-b6f7-4ed7-ae10-d2fefecd07e1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:36.023428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d2460-b6f7-4ed7-ae10-d2fefecd07e1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:36.025006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d2460-b6f7-4ed7-ae10-d2fefecd07e1 service swift] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:36.032932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 703/1402] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:05:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 19:05:36.173923 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b47c14-85e7-433e-b1c5-dfbf6f144d0c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:36.175440 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c1b47c14-85e7-433e-b1c5-dfbf6f144d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:36.227719 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:36.331257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b47c14-85e7-433e-b1c5-dfbf6f144d0c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:36.463635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2905781c-d0f8-473c-821d-cf31d4f042d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:36.464163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905781c-d0f8-473c-821d-cf31d4f042d2 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:36.464442 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905781c-d0f8-473c-821d-cf31d4f042d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:36.464710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2905781c-d0f8-473c-821d-cf31d4f042d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:36.466517 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2905781c-d0f8-473c-821d-cf31d4f042d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:36.505598 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:36.615387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905781c-d0f8-473c-821d-cf31d4f042d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:36.623107 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2905781c-d0f8-473c-821d-cf31d4f042d2 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:36.643474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 700/1403] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:05:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:05:38.591803 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1fecb-05ae-4058-9aa6-26c0d2928812 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:38.592444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1fecb-05ae-4058-9aa6-26c0d2928812 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:38.592444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1fecb-05ae-4058-9aa6-26c0d2928812 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:38.593874 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 704/1404] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:38.598858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15d1cf-ecf1-4c5e-852e-7dedf86477d8 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:38.599329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15d1cf-ecf1-4c5e-852e-7dedf86477d8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:38.599434 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c15d1cf-ecf1-4c5e-852e-7dedf86477d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:38.672214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0c15d1cf-ecf1-4c5e-852e-7dedf86477d8 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:38.673452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0c15d1cf-ecf1-4c5e-852e-7dedf86477d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:38.784267 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 701/1405] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:39.073319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1fecb-05ae-4058-9aa6-26c0d2928812 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:39.074979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e3a1fecb-05ae-4058-9aa6-26c0d2928812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:39.222403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:39.234581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1fecb-05ae-4058-9aa6-26c0d2928812 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:39.381988 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae02bc6-c6ce-46ce-82ad-21e04d187e8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:39.383337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae02bc6-c6ce-46ce-82ad-21e04d187e8e service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:39.383551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae02bc6-c6ce-46ce-82ad-21e04d187e8e service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:39.383830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae02bc6-c6ce-46ce-82ad-21e04d187e8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:39.385581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3ae02bc6-c6ce-46ce-82ad-21e04d187e8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:39.513970 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae02bc6-c6ce-46ce-82ad-21e04d187e8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:39.515954 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae02bc6-c6ce-46ce-82ad-21e04d187e8e service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:39.527834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:39.540237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 705/1406] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:05:41.567827 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8192ab-e3b2-4103-8179-46c06ca78af2 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:41.569264 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8192ab-e3b2-4103-8179-46c06ca78af2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:41.570062 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8192ab-e3b2-4103-8179-46c06ca78af2 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:41.572205 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 702/1407] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:05:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:41.577523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fb7f5-4c36-4c98-8bd3-4a0d3326332f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:41.577992 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fb7f5-4c36-4c98-8bd3-4a0d3326332f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:41.578102 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fb7f5-4c36-4c98-8bd3-4a0d3326332f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:41.652675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-5f0fb7f5-4c36-4c98-8bd3-4a0d3326332f None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:41.655184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5f0fb7f5-4c36-4c98-8bd3-4a0d3326332f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:41.765537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 706/1408] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:05:42.067403 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8192ab-e3b2-4103-8179-46c06ca78af2 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:05:42.068985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8a8192ab-e3b2-4103-8179-46c06ca78af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:42.072557 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:42.237610 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8192ab-e3b2-4103-8179-46c06ca78af2 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:05:42.319446 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:42.367509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568111e4-1c1d-4c6f-86a5-4a56d1967b64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:05:42.368120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568111e4-1c1d-4c6f-86a5-4a56d1967b64 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:42.368434 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568111e4-1c1d-4c6f-86a5-4a56d1967b64 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:42.368708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568111e4-1c1d-4c6f-86a5-4a56d1967b64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:42.370399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-568111e4-1c1d-4c6f-86a5-4a56d1967b64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:42.484766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568111e4-1c1d-4c6f-86a5-4a56d1967b64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:05:42.486463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568111e4-1c1d-4c6f-86a5-4a56d1967b64 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:05:42.506504 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 703/1409] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:05:51.763563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=67 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:52.506570 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=61 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:58.273759 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6571b6-adee-44c6-adb5-b8fd523ab25a None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:58.274336 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6571b6-adee-44c6-adb5-b8fd523ab25a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:58.274336 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6571b6-adee-44c6-adb5-b8fd523ab25a None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:58.275685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 707/1410] 10.222.0.65 () {54 vars in 918 bytes} [Wed Aug 31 19:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:58.281078 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb43864-4627-43af-b58c-26d0c57d30b8 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:58.281538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb43864-4627-43af-b58c-26d0c57d30b8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:58.281653 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb43864-4627-43af-b58c-26d0c57d30b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:58.357525 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-bcb43864-4627-43af-b58c-26d0c57d30b8 None None] MFA Rules not processed for user `74f1846e395e4d5483fcbb41eb28a77d`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:58.358717 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bcb43864-4627-43af-b58c-26d0c57d30b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:58.462685 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 704/1411] 10.222.0.65 () {58 vars in 1015 bytes} [Wed Aug 31 19:05:58 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:05:58.477593 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb0e25-abb5-4744-8763-42bad06c0419 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:58.477870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb0e25-abb5-4744-8763-42bad06c0419 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:58.478138 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adb0e25-abb5-4744-8763-42bad06c0419 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:58.479489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 708/1412] 10.222.0.65 () {54 vars in 918 bytes} [Wed Aug 31 19:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:05:58.483916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7a5d1-b758-4fd7-80f6-33ed8251216a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:05:58.484206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7a5d1-b758-4fd7-80f6-33ed8251216a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:05:58.484484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc7a5d1-b758-4fd7-80f6-33ed8251216a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:05:58.557209 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:05:58.560109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-1dc7a5d1-b758-4fd7-80f6-33ed8251216a None None] MFA Rules not processed for user `673a72ac9a9847e3812007b66ac7a5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:05:58.561864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1dc7a5d1-b758-4fd7-80f6-33ed8251216a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:05:58.675981 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 705/1413] 10.222.0.65 () {58 vars in 1015 bytes} [Wed Aug 31 19:05:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:01.063393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cae755-51bd-4cb4-91ca-ac931aa06796 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:01.063907 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cae755-51bd-4cb4-91ca-ac931aa06796 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:01.063907 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cae755-51bd-4cb4-91ca-ac931aa06796 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:01.065226 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:01.070308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c891a8c-19c3-4dd7-945f-d4b2704c4de8 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:01.070737 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c891a8c-19c3-4dd7-945f-d4b2704c4de8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:01.070833 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c891a8c-19c3-4dd7-945f-d4b2704c4de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:01.137725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9c891a8c-19c3-4dd7-945f-d4b2704c4de8 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:01.138918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9c891a8c-19c3-4dd7-945f-d4b2704c4de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:01.193968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:01.238161 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:01.522570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cae755-51bd-4cb4-91ca-ac931aa06796 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:01.525227 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-51cae755-51bd-4cb4-91ca-ac931aa06796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:01.670115 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cae755-51bd-4cb4-91ca-ac931aa06796 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:01.742598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:01.781910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d50c58-c722-4263-b123-fcbb67c1a669 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:01.782443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d50c58-c722-4263-b123-fcbb67c1a669 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:01.782719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d50c58-c722-4263-b123-fcbb67c1a669 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:01.782989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d50c58-c722-4263-b123-fcbb67c1a669 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:01.784566 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-59d50c58-c722-4263-b123-fcbb67c1a669 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:01.873828 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d50c58-c722-4263-b123-fcbb67c1a669 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:01.875417 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d50c58-c722-4263-b123-fcbb67c1a669 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:01.892713 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 710/1416] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:02.785784 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3af74-b391-42aa-86ae-49d464c9bfe5 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:02.786331 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3af74-b391-42aa-86ae-49d464c9bfe5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:02.786416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd3af74-b391-42aa-86ae-49d464c9bfe5 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:02.787723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:02.792788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8123a63-6c18-4b93-8b2e-e2feccc3c150 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:02.793230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8123a63-6c18-4b93-8b2e-e2feccc3c150 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:02.793343 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8123a63-6c18-4b93-8b2e-e2feccc3c150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:02.869107 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-d8123a63-6c18-4b93-8b2e-e2feccc3c150 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:02.870359 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d8123a63-6c18-4b93-8b2e-e2feccc3c150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:02.897288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:02.967960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:06:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:03.119167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3af74-b391-42aa-86ae-49d464c9bfe5 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:03.120632 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2fd3af74-b391-42aa-86ae-49d464c9bfe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:03.267715 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd3af74-b391-42aa-86ae-49d464c9bfe5 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:03.307343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:03.378527 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92eb98-7719-4fd1-99aa-63c3c75b467b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:03.379054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92eb98-7719-4fd1-99aa-63c3c75b467b service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:03.379323 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92eb98-7719-4fd1-99aa-63c3c75b467b service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:03.379586 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92eb98-7719-4fd1-99aa-63c3c75b467b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:03.381152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5f92eb98-7719-4fd1-99aa-63c3c75b467b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:03.467430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92eb98-7719-4fd1-99aa-63c3c75b467b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:03.468988 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92eb98-7719-4fd1-99aa-63c3c75b467b service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:03.487074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 708/1419] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:04.368785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce78cf6-2f9d-4db7-9fe0-368cb582bd07 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:04.369401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce78cf6-2f9d-4db7-9fe0-368cb582bd07 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:04.369401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce78cf6-2f9d-4db7-9fe0-368cb582bd07 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:04.370777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:04.375784 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169deee-5446-4fe7-a2c1-993af5f472c4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:04.376196 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169deee-5446-4fe7-a2c1-993af5f472c4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:04.376309 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d169deee-5446-4fe7-a2c1-993af5f472c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:04.429984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:04.452007 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d169deee-5446-4fe7-a2c1-993af5f472c4 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:04.453132 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d169deee-5446-4fe7-a2c1-993af5f472c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:04.549092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:06:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:04.701320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce78cf6-2f9d-4db7-9fe0-368cb582bd07 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:04.702993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-dce78cf6-2f9d-4db7-9fe0-368cb582bd07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:04.847483 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce78cf6-2f9d-4db7-9fe0-368cb582bd07 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:04.859390 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:04.959542 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5140dce-48a7-4ccf-a875-f6d062105a9e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:04.960395 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5140dce-48a7-4ccf-a875-f6d062105a9e service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:04.960817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5140dce-48a7-4ccf-a875-f6d062105a9e service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:04.961281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5140dce-48a7-4ccf-a875-f6d062105a9e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:04.963421 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c5140dce-48a7-4ccf-a875-f6d062105a9e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:05.052393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5140dce-48a7-4ccf-a875-f6d062105a9e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:05.054536 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5140dce-48a7-4ccf-a875-f6d062105a9e service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:05.072775 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 713/1422] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:05.931097 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12678382-27e7-47fa-8e51-eb3ddc88b9d8 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:05.931678 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12678382-27e7-47fa-8e51-eb3ddc88b9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:05.931678 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12678382-27e7-47fa-8e51-eb3ddc88b9d8 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:05.932978 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:05.937985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df9f53-2bfb-4ce9-9a29-1c9b82aa32dd None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:05.938400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df9f53-2bfb-4ce9-9a29-1c9b82aa32dd None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:05.938522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9df9f53-2bfb-4ce9-9a29-1c9b82aa32dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:05.958825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:06.015257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-d9df9f53-2bfb-4ce9-9a29-1c9b82aa32dd None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:06.016459 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d9df9f53-2bfb-4ce9-9a29-1c9b82aa32dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:06.111305 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:06:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:06.264221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12678382-27e7-47fa-8e51-eb3ddc88b9d8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:06.265784 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-12678382-27e7-47fa-8e51-eb3ddc88b9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:06.377292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:06.413353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12678382-27e7-47fa-8e51-eb3ddc88b9d8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:06.524131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113f0c7-9d56-401a-b00b-484281ab9b20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:06.524731 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113f0c7-9d56-401a-b00b-484281ab9b20 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:06.525010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113f0c7-9d56-401a-b00b-484281ab9b20 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:06.525334 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113f0c7-9d56-401a-b00b-484281ab9b20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:06.526912 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5113f0c7-9d56-401a-b00b-484281ab9b20 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:06.600835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:06.614355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113f0c7-9d56-401a-b00b-484281ab9b20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:06.615948 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113f0c7-9d56-401a-b00b-484281ab9b20 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:06.635071 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 711/1425] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:07.496018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0d5b6-e17c-4af3-bf95-d7ef4a12e33f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:07.496539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0d5b6-e17c-4af3-bf95-d7ef4a12e33f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:07.496539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e0d5b6-e17c-4af3-bf95-d7ef4a12e33f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:07.497885 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:07.502908 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa25a4a-0bd6-4d89-9587-227c4848f170 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:07.503325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa25a4a-0bd6-4d89-9587-227c4848f170 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:07.503456 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa25a4a-0bd6-4d89-9587-227c4848f170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:07.578781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-baa25a4a-0bd6-4d89-9587-227c4848f170 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:07.579970 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-baa25a4a-0bd6-4d89-9587-227c4848f170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:07.675868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:06:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:07.828454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0d5b6-e17c-4af3-bf95-d7ef4a12e33f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:07.829990 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-90e0d5b6-e17c-4af3-bf95-d7ef4a12e33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:07.911260 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:07.972637 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e0d5b6-e17c-4af3-bf95-d7ef4a12e33f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:08.082842 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40e7223-b022-4328-b2d2-bec7dfbe5894 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:08.083376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e7223-b022-4328-b2d2-bec7dfbe5894 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:08.083650 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e7223-b022-4328-b2d2-bec7dfbe5894 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:08.083918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40e7223-b022-4328-b2d2-bec7dfbe5894 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:08.085544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c40e7223-b022-4328-b2d2-bec7dfbe5894 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:08.134224 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:08.176314 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e7223-b022-4328-b2d2-bec7dfbe5894 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:08.177949 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40e7223-b022-4328-b2d2-bec7dfbe5894 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:08.197775 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 716/1428] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:09.008791 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832d6fa-bf5a-405d-8282-44f40e6c039b None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:09.009386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832d6fa-bf5a-405d-8282-44f40e6c039b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:09.009386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832d6fa-bf5a-405d-8282-44f40e6c039b None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:09.010722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:06:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:09.015540 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf39af-6214-4d4c-85b7-8e50bd10c99e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:09.016093 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf39af-6214-4d4c-85b7-8e50bd10c99e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:09.016093 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdf39af-6214-4d4c-85b7-8e50bd10c99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:09.091704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-1cdf39af-6214-4d4c-85b7-8e50bd10c99e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:09.092834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1cdf39af-6214-4d4c-85b7-8e50bd10c99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:09.186973 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:06:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:09.337822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832d6fa-bf5a-405d-8282-44f40e6c039b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:09.339358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2832d6fa-bf5a-405d-8282-44f40e6c039b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:09.388025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:09.483319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832d6fa-bf5a-405d-8282-44f40e6c039b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:09.595973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48396012-f91a-4d89-990d-433acc16e839 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:09.596574 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48396012-f91a-4d89-990d-433acc16e839 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:09.596853 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48396012-f91a-4d89-990d-433acc16e839 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:09.597124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48396012-f91a-4d89-990d-433acc16e839 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:09.598747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-48396012-f91a-4d89-990d-433acc16e839 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:09.606229 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:09.690771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48396012-f91a-4d89-990d-433acc16e839 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:09.692305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48396012-f91a-4d89-990d-433acc16e839 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:09.709973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 714/1431] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:10.542060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd731c6-c90f-4ec0-8d23-7ec581b794de None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:10.542697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd731c6-c90f-4ec0-8d23-7ec581b794de None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:10.542697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd731c6-c90f-4ec0-8d23-7ec581b794de None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:10.544024 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:10.548847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862b736-4d8a-4e9b-ab62-3dee52f27038 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:10.549332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862b736-4d8a-4e9b-ab62-3dee52f27038 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:10.549442 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2862b736-4d8a-4e9b-ab62-3dee52f27038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:10.626788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2862b736-4d8a-4e9b-ab62-3dee52f27038 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:10.627977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2862b736-4d8a-4e9b-ab62-3dee52f27038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:10.693756 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:10.723544 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:10.873792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd731c6-c90f-4ec0-8d23-7ec581b794de None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:10.875337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2cd731c6-c90f-4ec0-8d23-7ec581b794de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:10.893122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:11.023165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd731c6-c90f-4ec0-8d23-7ec581b794de None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:11.110288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:11.137607 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014729f-7c1a-4a67-917c-31617daf1e96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:11.138215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014729f-7c1a-4a67-917c-31617daf1e96 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:11.138493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014729f-7c1a-4a67-917c-31617daf1e96 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:11.138762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014729f-7c1a-4a67-917c-31617daf1e96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:11.140392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a014729f-7c1a-4a67-917c-31617daf1e96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:11.232379 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014729f-7c1a-4a67-917c-31617daf1e96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:11.234006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a014729f-7c1a-4a67-917c-31617daf1e96 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:11.251820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 719/1434] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:12.050956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732e554-4c16-487b-94b7-d34346f437fe None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:12.051437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732e554-4c16-487b-94b7-d34346f437fe None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:12.051550 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8732e554-4c16-487b-94b7-d34346f437fe None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:12.052910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:06:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:12.058432 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d43cc-82d0-4556-9f2b-4516883de9fd None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:12.058844 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d43cc-82d0-4556-9f2b-4516883de9fd None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:12.058973 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12d43cc-82d0-4556-9f2b-4516883de9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:12.133879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c12d43cc-82d0-4556-9f2b-4516883de9fd None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:12.135083 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c12d43cc-82d0-4556-9f2b-4516883de9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:12.175597 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:12.234141 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:12.385802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732e554-4c16-487b-94b7-d34346f437fe None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:12.387252 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8732e554-4c16-487b-94b7-d34346f437fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:12.532961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8732e554-4c16-487b-94b7-d34346f437fe None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:12.587254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:12.646094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d5645-72e7-449e-8796-fc0cd81c1856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:12.646631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d5645-72e7-449e-8796-fc0cd81c1856 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:12.646910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d5645-72e7-449e-8796-fc0cd81c1856 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:12.647184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d5645-72e7-449e-8796-fc0cd81c1856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:12.648868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a03d5645-72e7-449e-8796-fc0cd81c1856 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:12.736327 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d5645-72e7-449e-8796-fc0cd81c1856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:12.737963 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d5645-72e7-449e-8796-fc0cd81c1856 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:12.756261 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 717/1437] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:13.560550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77ea78-0a2a-4f72-94d7-6447ff16c270 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:13.561105 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77ea78-0a2a-4f72-94d7-6447ff16c270 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:13.561105 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77ea78-0a2a-4f72-94d7-6447ff16c270 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:13.562493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:13.567639 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f507a-f24f-4f33-96bb-231ab51e0bbc None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:13.568049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f507a-f24f-4f33-96bb-231ab51e0bbc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:13.568188 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0f507a-f24f-4f33-96bb-231ab51e0bbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:13.645977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ee0f507a-f24f-4f33-96bb-231ab51e0bbc None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:13.647305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ee0f507a-f24f-4f33-96bb-231ab51e0bbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:13.650071 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:13.860069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:06:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:14.024622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77ea78-0a2a-4f72-94d7-6447ff16c270 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:14.026188 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-eb77ea78-0a2a-4f72-94d7-6447ff16c270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:14.169751 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77ea78-0a2a-4f72-94d7-6447ff16c270 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:14.197452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:14.286614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ba7435-8d1e-4871-99b5-67cb710d0881 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:14.287216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba7435-8d1e-4871-99b5-67cb710d0881 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:14.287494 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba7435-8d1e-4871-99b5-67cb710d0881 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:14.287762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba7435-8d1e-4871-99b5-67cb710d0881 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:14.290788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e8ba7435-8d1e-4871-99b5-67cb710d0881 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:14.382974 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba7435-8d1e-4871-99b5-67cb710d0881 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:14.384556 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ba7435-8d1e-4871-99b5-67cb710d0881 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:14.402563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 722/1440] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:16.209310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3fc48-f0fa-48f7-9623-ef189c8790b8 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:16.209796 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3fc48-f0fa-48f7-9623-ef189c8790b8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:16.209902 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a3fc48-f0fa-48f7-9623-ef189c8790b8 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:16.211714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:16.216955 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cf4b6-6171-4ebc-aa89-5fc854e0b4f9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:16.217675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cf4b6-6171-4ebc-aa89-5fc854e0b4f9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:16.217675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149cf4b6-6171-4ebc-aa89-5fc854e0b4f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:16.258277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:16.284099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-149cf4b6-6171-4ebc-aa89-5fc854e0b4f9 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:16.285343 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-149cf4b6-6171-4ebc-aa89-5fc854e0b4f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:16.380716 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:06:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:16.669732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3fc48-f0fa-48f7-9623-ef189c8790b8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:16.671245 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f0a3fc48-f0fa-48f7-9623-ef189c8790b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:16.773933 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a3fc48-f0fa-48f7-9623-ef189c8790b8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:16.805363 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:16.847221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d057ee-722a-40f1-87fb-4202236cdc52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:16.847840 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d057ee-722a-40f1-87fb-4202236cdc52 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:16.848118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d057ee-722a-40f1-87fb-4202236cdc52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:16.848387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d057ee-722a-40f1-87fb-4202236cdc52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:16.850001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-45d057ee-722a-40f1-87fb-4202236cdc52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:16.956444 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d057ee-722a-40f1-87fb-4202236cdc52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:16.957820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d057ee-722a-40f1-87fb-4202236cdc52 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:16.975588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 720/1443] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:19.021026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d7820-afc7-44ad-855e-5d4db7338bf3 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:19.021586 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d7820-afc7-44ad-855e-5d4db7338bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:19.021586 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8d7820-afc7-44ad-855e-5d4db7338bf3 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:19.022996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:06:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:19.028266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc019d-50d6-46f0-a2c3-f9c7ad6420ed None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:19.028677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc019d-50d6-46f0-a2c3-f9c7ad6420ed None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:19.028808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc019d-50d6-46f0-a2c3-f9c7ad6420ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:19.096131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a6dc019d-50d6-46f0-a2c3-f9c7ad6420ed None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:19.097281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a6dc019d-50d6-46f0-a2c3-f9c7ad6420ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:19.110513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:19.194212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:06:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:19.476423 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d7820-afc7-44ad-855e-5d4db7338bf3 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:19.478030 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8a8d7820-afc7-44ad-855e-5d4db7338bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:19.582366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8d7820-afc7-44ad-855e-5d4db7338bf3 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:19.594583 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:19.657893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4abddf-74d8-452d-814e-594a63a0cee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:19.658493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4abddf-74d8-452d-814e-594a63a0cee6 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:19.658768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4abddf-74d8-452d-814e-594a63a0cee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:19.659035 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4abddf-74d8-452d-814e-594a63a0cee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:19.660635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0e4abddf-74d8-452d-814e-594a63a0cee6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:19.770372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4abddf-74d8-452d-814e-594a63a0cee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:19.771827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4abddf-74d8-452d-814e-594a63a0cee6 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:19.790072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 725/1446] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:22.009773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85edcb-67a4-47d2-822f-b0e4b4ef8b38 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:22.010354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85edcb-67a4-47d2-822f-b0e4b4ef8b38 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:22.010354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d85edcb-67a4-47d2-822f-b0e4b4ef8b38 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:22.011633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:22.016824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b1d90-ac45-4a61-a8a1-b3623e737e86 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:22.017279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b1d90-ac45-4a61-a8a1-b3623e737e86 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:22.017425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b1d90-ac45-4a61-a8a1-b3623e737e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:22.065766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:22.085048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-925b1d90-ac45-4a61-a8a1-b3623e737e86 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:22.086281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-925b1d90-ac45-4a61-a8a1-b3623e737e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:22.180200 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:06:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:22.461411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85edcb-67a4-47d2-822f-b0e4b4ef8b38 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:22.462931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7d85edcb-67a4-47d2-822f-b0e4b4ef8b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:22.564360 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d85edcb-67a4-47d2-822f-b0e4b4ef8b38 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:22.604407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:22.638164 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454ea22-5272-40d4-878d-46b8d3723951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:22.638759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454ea22-5272-40d4-878d-46b8d3723951 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:22.639038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454ea22-5272-40d4-878d-46b8d3723951 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:22.639308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454ea22-5272-40d4-878d-46b8d3723951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:22.640896 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9454ea22-5272-40d4-878d-46b8d3723951 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:22.746816 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454ea22-5272-40d4-878d-46b8d3723951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:22.748199 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454ea22-5272-40d4-878d-46b8d3723951 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:22.766134 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 723/1449] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:24.794401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de8d2c-fa83-4c71-98dc-ec4501dae368 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.794947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de8d2c-fa83-4c71-98dc-ec4501dae368 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.794947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63de8d2c-fa83-4c71-98dc-ec4501dae368 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.796293 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:24.801884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397648d8-f7fb-4d0c-b407-709568aad7d2 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:24.802267 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397648d8-f7fb-4d0c-b407-709568aad7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:24.802433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397648d8-f7fb-4d0c-b407-709568aad7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:24.867143 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-397648d8-f7fb-4d0c-b407-709568aad7d2 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:24.868326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-397648d8-f7fb-4d0c-b407-709568aad7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:24.888811 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:24.965761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:25.255794 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de8d2c-fa83-4c71-98dc-ec4501dae368 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:25.257406 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-63de8d2c-fa83-4c71-98dc-ec4501dae368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:25.361949 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63de8d2c-fa83-4c71-98dc-ec4501dae368 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:25.381816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:25.434677 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb29375-73e1-45d0-a508-a58110537a32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:25.435270 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb29375-73e1-45d0-a508-a58110537a32 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:25.435544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb29375-73e1-45d0-a508-a58110537a32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:25.435809 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb29375-73e1-45d0-a508-a58110537a32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:25.437419 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cbb29375-73e1-45d0-a508-a58110537a32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:25.544240 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb29375-73e1-45d0-a508-a58110537a32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:25.545710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb29375-73e1-45d0-a508-a58110537a32 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:25.563333 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 728/1452] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:27.672528 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645c4aa-b487-42c3-b754-56d327a94e2d None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:27.673102 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645c4aa-b487-42c3-b754-56d327a94e2d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:27.673102 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1645c4aa-b487-42c3-b754-56d327a94e2d None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:27.674460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:27.679650 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150cef40-e993-4617-817e-786aff444acf None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:27.680051 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150cef40-e993-4617-817e-786aff444acf None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:27.680192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150cef40-e993-4617-817e-786aff444acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:27.743489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:27.746646 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-150cef40-e993-4617-817e-786aff444acf None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:27.748446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-150cef40-e993-4617-817e-786aff444acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:27.844857 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:28.126240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645c4aa-b487-42c3-b754-56d327a94e2d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:28.127800 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1645c4aa-b487-42c3-b754-56d327a94e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:28.234213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1645c4aa-b487-42c3-b754-56d327a94e2d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:28.282534 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:28.306874 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835a0f8-26df-4ebe-bd21-e56f8dc87a5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:28.307490 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835a0f8-26df-4ebe-bd21-e56f8dc87a5c service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:28.307763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835a0f8-26df-4ebe-bd21-e56f8dc87a5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:28.308026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835a0f8-26df-4ebe-bd21-e56f8dc87a5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:28.309620 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6835a0f8-26df-4ebe-bd21-e56f8dc87a5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:28.416254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6835a0f8-26df-4ebe-bd21-e56f8dc87a5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:28.417624 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6835a0f8-26df-4ebe-bd21-e56f8dc87a5c service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:28.434936 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 726/1455] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:30.388566 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9e682-5871-4aea-a3bb-f60011032063 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:30.389145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9e682-5871-4aea-a3bb-f60011032063 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:30.389145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9e682-5871-4aea-a3bb-f60011032063 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:30.390521 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:30.395806 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f5741-fe2a-4fe4-b94e-27bd85379bfc None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:30.396274 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f5741-fe2a-4fe4-b94e-27bd85379bfc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:30.396389 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f5741-fe2a-4fe4-b94e-27bd85379bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:30.461498 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4d2f5741-fe2a-4fe4-b94e-27bd85379bfc None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:30.462609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4d2f5741-fe2a-4fe4-b94e-27bd85379bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:30.491336 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:30.558771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:06:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:30.844503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9e682-5871-4aea-a3bb-f60011032063 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:30.846073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-15d9e682-5871-4aea-a3bb-f60011032063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:30.949410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9e682-5871-4aea-a3bb-f60011032063 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:30.977351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:31.022086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6229684-9f2e-4d29-97ba-fd3e05fe991a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:31.022681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6229684-9f2e-4d29-97ba-fd3e05fe991a service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:31.022979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6229684-9f2e-4d29-97ba-fd3e05fe991a service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:31.023234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6229684-9f2e-4d29-97ba-fd3e05fe991a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:31.024818 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f6229684-9f2e-4d29-97ba-fd3e05fe991a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:31.137935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6229684-9f2e-4d29-97ba-fd3e05fe991a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:31.139364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6229684-9f2e-4d29-97ba-fd3e05fe991a service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:31.157711 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 731/1458] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:33.110826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2769bd1-45c8-48ed-9310-068ec3e59b52 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:33.111362 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2769bd1-45c8-48ed-9310-068ec3e59b52 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:33.111362 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2769bd1-45c8-48ed-9310-068ec3e59b52 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:33.112653 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:06:33.118634 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36922825-9129-499e-bca1-247dd718b429 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:33.119040 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36922825-9129-499e-bca1-247dd718b429 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:33.119178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36922825-9129-499e-bca1-247dd718b429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:33.184720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-36922825-9129-499e-bca1-247dd718b429 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:33.186021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-36922825-9129-499e-bca1-247dd718b429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:33.195072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:33.282706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:06:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:33.596750 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2769bd1-45c8-48ed-9310-068ec3e59b52 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:33.598354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c2769bd1-45c8-48ed-9310-068ec3e59b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:33.702256 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2769bd1-45c8-48ed-9310-068ec3e59b52 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:33.757207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:33.775308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1c6cc-5f38-497b-9903-586edc16d14a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:33.775914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1c6cc-5f38-497b-9903-586edc16d14a service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:33.776188 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1c6cc-5f38-497b-9903-586edc16d14a service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:33.776455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1c6cc-5f38-497b-9903-586edc16d14a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:33.778107 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-90a1c6cc-5f38-497b-9903-586edc16d14a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:33.887229 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1c6cc-5f38-497b-9903-586edc16d14a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:33.888570 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1c6cc-5f38-497b-9903-586edc16d14a service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:33.906531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 729/1461] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:35.827939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0da64d-b3aa-4ce2-b163-f30bd144f626 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:35.828485 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0da64d-b3aa-4ce2-b163-f30bd144f626 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:35.828485 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0da64d-b3aa-4ce2-b163-f30bd144f626 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:35.829927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:35.836206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab81a6d-b1df-4328-bfff-0d42f6f29269 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:35.836641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab81a6d-b1df-4328-bfff-0d42f6f29269 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:35.836885 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab81a6d-b1df-4328-bfff-0d42f6f29269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:35.903551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-cab81a6d-b1df-4328-bfff-0d42f6f29269 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:35.904763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cab81a6d-b1df-4328-bfff-0d42f6f29269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:35.942111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:35.999074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:36.282962 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0da64d-b3aa-4ce2-b163-f30bd144f626 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:36.284495 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5c0da64d-b3aa-4ce2-b163-f30bd144f626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:36.388540 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0da64d-b3aa-4ce2-b163-f30bd144f626 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:36.425072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:36.461483 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173e7ab-c7fa-4ad3-be5c-67abbecf1576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:36.462018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173e7ab-c7fa-4ad3-be5c-67abbecf1576 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:36.462296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173e7ab-c7fa-4ad3-be5c-67abbecf1576 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:36.462567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173e7ab-c7fa-4ad3-be5c-67abbecf1576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:36.464178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9173e7ab-c7fa-4ad3-be5c-67abbecf1576 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:36.573748 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173e7ab-c7fa-4ad3-be5c-67abbecf1576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:36.575478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173e7ab-c7fa-4ad3-be5c-67abbecf1576 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:36.594686 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 734/1464] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:38.498199 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055c913-7ac9-4c63-bba7-1fa85fae0b27 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:38.498725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055c913-7ac9-4c63-bba7-1fa85fae0b27 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:38.498725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e055c913-7ac9-4c63-bba7-1fa85fae0b27 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:38.499995 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:06:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:38.505147 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cddaec-f602-4d65-b1fa-b75a77e0b5da None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:38.505559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cddaec-f602-4d65-b1fa-b75a77e0b5da None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:38.505732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cddaec-f602-4d65-b1fa-b75a77e0b5da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:38.570813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-62cddaec-f602-4d65-b1fa-b75a77e0b5da None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:38.571989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-62cddaec-f602-4d65-b1fa-b75a77e0b5da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:38.589836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:38.672931 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:38.954923 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055c913-7ac9-4c63-bba7-1fa85fae0b27 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:38.956460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e055c913-7ac9-4c63-bba7-1fa85fae0b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:39.057777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e055c913-7ac9-4c63-bba7-1fa85fae0b27 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:39.122848 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:39.129729 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab04a43-778f-413f-8816-e49ed26cbb38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:39.130335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab04a43-778f-413f-8816-e49ed26cbb38 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:39.130611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab04a43-778f-413f-8816-e49ed26cbb38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:39.131401 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab04a43-778f-413f-8816-e49ed26cbb38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:39.133000 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4ab04a43-778f-413f-8816-e49ed26cbb38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:39.243187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab04a43-778f-413f-8816-e49ed26cbb38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:39.244868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab04a43-778f-413f-8816-e49ed26cbb38 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:39.262985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 732/1467] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:41.170054 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd2fa5-af5d-4d73-823c-7c566fc871a8 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:41.170749 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd2fa5-af5d-4d73-823c-7c566fc871a8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:41.170749 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dd2fa5-af5d-4d73-823c-7c566fc871a8 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:41.171951 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:06:41.177476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdb11f-eb34-4755-874b-4562d3e26ded None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:41.177936 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdb11f-eb34-4755-874b-4562d3e26ded None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:41.178043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdb11f-eb34-4755-874b-4562d3e26ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:41.244183 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-fffdb11f-eb34-4755-874b-4562d3e26ded None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:06:41.245665 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-fffdb11f-eb34-4755-874b-4562d3e26ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:41.293970 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:41.341603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:06:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:06:41.621897 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd2fa5-af5d-4d73-823c-7c566fc871a8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:06:41.623429 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-02dd2fa5-af5d-4d73-823c-7c566fc871a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:41.725834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dd2fa5-af5d-4d73-823c-7c566fc871a8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:06:41.773939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:41.799459 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2434616-b327-4cd5-be0e-0752bbf54949 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:06:41.800057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2434616-b327-4cd5-be0e-0752bbf54949 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:06:41.800330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2434616-b327-4cd5-be0e-0752bbf54949 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:06:41.800597 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2434616-b327-4cd5-be0e-0752bbf54949 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:06:41.802215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c2434616-b327-4cd5-be0e-0752bbf54949 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:06:41.916796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2434616-b327-4cd5-be0e-0752bbf54949 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:06:41.918274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2434616-b327-4cd5-be0e-0752bbf54949 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:06:41.936894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 737/1470] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:06:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:06:51.339812 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=22 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:06:51.934493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=66 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:01.494421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a1c9d-0c81-4ff5-bc61-4df86d389b15 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:01.495090 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a1c9d-0c81-4ff5-bc61-4df86d389b15 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:01.495090 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3a1c9d-0c81-4ff5-bc61-4df86d389b15 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:01.496273 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 734/1471] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:01.501592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100fff6-e385-415c-af3c-f98bd5cdc30c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:01.501992 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100fff6-e385-415c-af3c-f98bd5cdc30c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:01.502126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1100fff6-e385-415c-af3c-f98bd5cdc30c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:01.568467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-1100fff6-e385-415c-af3c-f98bd5cdc30c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:01.569705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1100fff6-e385-415c-af3c-f98bd5cdc30c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:01.664866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 738/1472] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:07:01.964588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335906dd-faa6-46cb-9abc-b310a4076490 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:01.964871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335906dd-faa6-46cb-9abc-b310a4076490 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:01.965163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335906dd-faa6-46cb-9abc-b310a4076490 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:01.966526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 735/1473] 10.222.0.65 () {54 vars in 967 bytes} [Wed Aug 31 19:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:01.971279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0455f-b555-4640-9784-c256c536477c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:01.971565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0455f-b555-4640-9784-c256c536477c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:01.971840 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc0455f-b555-4640-9784-c256c536477c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:02.053585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3dc0455f-b555-4640-9784-c256c536477c None None] MFA Rules not processed for user `1863336fecad4987acdfd0b3f7b2f70f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:02.055319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3dc0455f-b555-4640-9784-c256c536477c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:02.077247 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:02.133052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 739/1474] 10.222.0.65 () {58 vars in 1064 bytes} [Wed Aug 31 19:07:01 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:07:02.140544 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335906dd-faa6-46cb-9abc-b310a4076490 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:02.142099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-335906dd-faa6-46cb-9abc-b310a4076490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:02.247147 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335906dd-faa6-46cb-9abc-b310a4076490 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:02.323770 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36551137-c214-40bf-ac86-2f326901b045 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1863336fecad4987acdfd0b3f7b2f70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:02.324365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36551137-c214-40bf-ac86-2f326901b045 service glance] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:02.324644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36551137-c214-40bf-ac86-2f326901b045 service glance] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:02.324913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36551137-c214-40bf-ac86-2f326901b045 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:02.326543 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-36551137-c214-40bf-ac86-2f326901b045 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:02.381888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:02.436821 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36551137-c214-40bf-ac86-2f326901b045 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:02.438285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36551137-c214-40bf-ac86-2f326901b045 service glance] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:02.456337 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 736/1475] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:07:04.443888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ae9ba-4219-491e-9d35-81865c92137a None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:04.444451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ae9ba-4219-491e-9d35-81865c92137a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:04.444451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ae9ba-4219-491e-9d35-81865c92137a None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:04.445857 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 740/1476] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:04.451225 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e069b7-b628-4a5d-bd17-7249e29f8cb8 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:04.451622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e069b7-b628-4a5d-bd17-7249e29f8cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:04.451723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e069b7-b628-4a5d-bd17-7249e29f8cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:04.516683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-86e069b7-b628-4a5d-bd17-7249e29f8cb8 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:04.517910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-86e069b7-b628-4a5d-bd17-7249e29f8cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:04.612760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 737/1477] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:07:04.893265 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ae9ba-4219-491e-9d35-81865c92137a None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:04.894875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a20ae9ba-4219-491e-9d35-81865c92137a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:05.017900 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20ae9ba-4219-491e-9d35-81865c92137a None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:05.064376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:05.108398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0ed66-2ac8-4701-9d96-f9547fea876f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:05.108914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0ed66-2ac8-4701-9d96-f9547fea876f admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:05.109181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0ed66-2ac8-4701-9d96-f9547fea876f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:05.109465 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0ed66-2ac8-4701-9d96-f9547fea876f admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:05.110086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0ed66-2ac8-4701-9d96-f9547fea876f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:05.111826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: WARNING py.warnings [None req-f6a0ed66-2ac8-4701-9d96-f9547fea876f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:07:05.111826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: warnings.warn(msg) Aug 31 19:07:05.111826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: Aug 31 19:07:05.112959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0ed66-2ac8-4701-9d96-f9547fea876f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:05.132675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 741/1478] 10.222.0.65 () {56 vars in 1108 bytes} [Wed Aug 31 19:07:04 2022] GET /identity/v3/projects => generated 2553 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:07:07.064597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3afc9-3a98-429c-8589-5f401da00aa5 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:07.065203 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3afc9-3a98-429c-8589-5f401da00aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:07.065203 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3afc9-3a98-429c-8589-5f401da00aa5 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:07.066508 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 738/1479] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:07.071848 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a12f6e-a1ad-4eb4-b9f0-d8a53e96f388 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:07.072257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a12f6e-a1ad-4eb4-b9f0-d8a53e96f388 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:07.072358 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a12f6e-a1ad-4eb4-b9f0-d8a53e96f388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:07.138553 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f3a12f6e-a1ad-4eb4-b9f0-d8a53e96f388 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:07.139769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f3a12f6e-a1ad-4eb4-b9f0-d8a53e96f388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:07.235796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 742/1480] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:07:07.530861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3afc9-3a98-429c-8589-5f401da00aa5 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:07.532386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e4f3afc9-3a98-429c-8589-5f401da00aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:07.542027 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:07.674934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3afc9-3a98-429c-8589-5f401da00aa5 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:07.758656 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:07.791297 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d546ed-39ee-4dd5-87a8-62f61e790f71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:07.791822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d546ed-39ee-4dd5-87a8-62f61e790f71 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:07.792096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d546ed-39ee-4dd5-87a8-62f61e790f71 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:07.792359 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d546ed-39ee-4dd5-87a8-62f61e790f71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:07.793969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-58d546ed-39ee-4dd5-87a8-62f61e790f71 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:07.883117 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d546ed-39ee-4dd5-87a8-62f61e790f71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:07.884698 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d546ed-39ee-4dd5-87a8-62f61e790f71 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:07.902240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 739/1481] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:07:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:07:09.976493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbae31-0e35-4d75-b125-6e8a0f8357cb None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:09.977073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbae31-0e35-4d75-b125-6e8a0f8357cb None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:09.977073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefbae31-0e35-4d75-b125-6e8a0f8357cb None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:09.978431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 743/1482] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:09.983790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8d26f-6751-4b0c-8750-966a0248a2ae None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:09.984269 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8d26f-6751-4b0c-8750-966a0248a2ae None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:09.984374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8d26f-6751-4b0c-8750-966a0248a2ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:10.049886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ddc8d26f-6751-4b0c-8750-966a0248a2ae None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:10.051086 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ddc8d26f-6751-4b0c-8750-966a0248a2ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:10.096644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:10.147427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 740/1483] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:07:10.434419 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbae31-0e35-4d75-b125-6e8a0f8357cb None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:10.435960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cefbae31-0e35-4d75-b125-6e8a0f8357cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:10.460383 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:10.539287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefbae31-0e35-4d75-b125-6e8a0f8357cb None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:10.612840 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369875b4-7d4e-4eb6-82c7-a3063ef5874e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:10.613465 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369875b4-7d4e-4eb6-82c7-a3063ef5874e service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:10.613745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369875b4-7d4e-4eb6-82c7-a3063ef5874e service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:10.614036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369875b4-7d4e-4eb6-82c7-a3063ef5874e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:10.615645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-369875b4-7d4e-4eb6-82c7-a3063ef5874e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:10.688696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:10.724452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369875b4-7d4e-4eb6-82c7-a3063ef5874e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:10.725915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369875b4-7d4e-4eb6-82c7-a3063ef5874e service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:10.743995 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 744/1484] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:07:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:07:12.762074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a789ddb-3a39-4e69-88a8-c7a2e03f3248 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:12.762650 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a789ddb-3a39-4e69-88a8-c7a2e03f3248 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:12.762650 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a789ddb-3a39-4e69-88a8-c7a2e03f3248 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:12.763939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 741/1485] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:12.768963 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893ebd1-d6e0-4067-8413-9150a7d4b7b0 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:12.769471 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893ebd1-d6e0-4067-8413-9150a7d4b7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:12.769575 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893ebd1-d6e0-4067-8413-9150a7d4b7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:12.838777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-5893ebd1-d6e0-4067-8413-9150a7d4b7b0 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:12.840005 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5893ebd1-d6e0-4067-8413-9150a7d4b7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:12.943999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 745/1486] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:07:13.230888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a789ddb-3a39-4e69-88a8-c7a2e03f3248 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:13.232425 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6a789ddb-3a39-4e69-88a8-c7a2e03f3248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:13.334896 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a789ddb-3a39-4e69-88a8-c7a2e03f3248 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:13.404648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:13.406433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a3d04-f8e6-4fd2-af0a-4b280f9f81d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:13.407206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a3d04-f8e6-4fd2-af0a-4b280f9f81d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:13.407499 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a3d04-f8e6-4fd2-af0a-4b280f9f81d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:13.407983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a3d04-f8e6-4fd2-af0a-4b280f9f81d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:13.409959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3e2a3d04-f8e6-4fd2-af0a-4b280f9f81d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:13.517396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a3d04-f8e6-4fd2-af0a-4b280f9f81d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:13.518821 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a3d04-f8e6-4fd2-af0a-4b280f9f81d5 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:13.537731 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 742/1487] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:07:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:07:15.582759 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40863ad-7b7a-4643-b5f2-cea3145c8dd6 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:15.583329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40863ad-7b7a-4643-b5f2-cea3145c8dd6 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:15.583329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40863ad-7b7a-4643-b5f2-cea3145c8dd6 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:15.584638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 746/1488] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:15.589880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23818474-3162-4806-b556-77d31d4788c6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:15.590293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23818474-3162-4806-b556-77d31d4788c6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:15.590417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23818474-3162-4806-b556-77d31d4788c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:15.656315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-23818474-3162-4806-b556-77d31d4788c6 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:15.657523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-23818474-3162-4806-b556-77d31d4788c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:15.710266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:15.755475 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 743/1489] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:07:16.051159 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40863ad-7b7a-4643-b5f2-cea3145c8dd6 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:16.053324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e40863ad-7b7a-4643-b5f2-cea3145c8dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:16.088557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:16.157833 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40863ad-7b7a-4643-b5f2-cea3145c8dd6 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:16.235151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056c044-4631-44e6-b30a-3707afb347e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:16.235753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056c044-4631-44e6-b30a-3707afb347e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:16.236032 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056c044-4631-44e6-b30a-3707afb347e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:16.236303 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056c044-4631-44e6-b30a-3707afb347e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:16.237941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f056c044-4631-44e6-b30a-3707afb347e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:16.433467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:16.463513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f056c044-4631-44e6-b30a-3707afb347e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:16.464938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f056c044-4631-44e6-b30a-3707afb347e3 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:16.483111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 747/1490] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:07:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:07:18.629631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa7063-7863-4cf5-a925-4f2f3102d3b8 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:18.630227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa7063-7863-4cf5-a925-4f2f3102d3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:18.630227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa7063-7863-4cf5-a925-4f2f3102d3b8 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:18.631518 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 744/1491] 10.222.0.65 () {54 vars in 927 bytes} [Wed Aug 31 19:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:07:18.636693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fb08a-295d-431b-ab5a-9f794e009e12 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:18.637115 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fb08a-295d-431b-ab5a-9f794e009e12 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:18.637270 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535fb08a-295d-431b-ab5a-9f794e009e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:18.701429 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-535fb08a-295d-431b-ab5a-9f794e009e12 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:07:18.702605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-535fb08a-295d-431b-ab5a-9f794e009e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:18.795056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 748/1492] 10.222.0.65 () {58 vars in 1024 bytes} [Wed Aug 31 19:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:07:19.092569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa7063-7863-4cf5-a925-4f2f3102d3b8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:07:19.094132 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-01fa7063-7863-4cf5-a925-4f2f3102d3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:19.196770 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa7063-7863-4cf5-a925-4f2f3102d3b8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:07:19.271072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a48d6-4423-4bb3-a356-9a754915388d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:07:19.271672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a48d6-4423-4bb3-a356-9a754915388d service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:07:19.271947 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a48d6-4423-4bb3-a356-9a754915388d service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:07:19.272214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a48d6-4423-4bb3-a356-9a754915388d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:07:19.273810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-350a48d6-4423-4bb3-a356-9a754915388d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:07:19.282714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:19.380564 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a48d6-4423-4bb3-a356-9a754915388d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:07:19.381934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a48d6-4423-4bb3-a356-9a754915388d service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:07:19.399946 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 745/1493] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:07:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:07:28.793298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=82 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:07:29.397636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=50 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:31.591059 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab08f02-16cb-48bf-8cd3-1336530e11e0 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:31.591675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab08f02-16cb-48bf-8cd3-1336530e11e0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:31.591675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab08f02-16cb-48bf-8cd3-1336530e11e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:31.665425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8ab08f02-16cb-48bf-8cd3-1336530e11e0 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:08:31.666612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8ab08f02-16cb-48bf-8cd3-1336530e11e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:31.760454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 749/1494] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:08:31.769063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6347d-9a20-4cab-b0ba-3a35e0cf27bb None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:31.769578 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6347d-9a20-4cab-b0ba-3a35e0cf27bb None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:31.769684 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e6347d-9a20-4cab-b0ba-3a35e0cf27bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:31.844197 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-70e6347d-9a20-4cab-b0ba-3a35e0cf27bb None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:08:31.845337 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-70e6347d-9a20-4cab-b0ba-3a35e0cf27bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:31.938630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 746/1495] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:08:31.943965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab08f02-16cb-48bf-8cd3-1336530e11e0 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:31.945164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8ab08f02-16cb-48bf-8cd3-1336530e11e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:31.956527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab08f02-16cb-48bf-8cd3-1336530e11e0 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:32.018630 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:32.051855 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fd4e1-7c22-47e5-915f-30a0bca2f2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:32.052461 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fd4e1-7c22-47e5-915f-30a0bca2f2cd admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:32.052733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fd4e1-7c22-47e5-915f-30a0bca2f2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:32.052998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11fd4e1-7c22-47e5-915f-30a0bca2f2cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:32.053797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fd4e1-7c22-47e5-915f-30a0bca2f2cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:08:32.054089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fd4e1-7c22-47e5-915f-30a0bca2f2cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:32.055091 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: WARNING py.warnings [None req-b11fd4e1-7c22-47e5-915f-30a0bca2f2cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:08:32.055091 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: warnings.warn(msg) Aug 31 19:08:32.055091 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: Aug 31 19:08:32.055965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11fd4e1-7c22-47e5-915f-30a0bca2f2cd admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:32.067976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 750/1496] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:08:32.076427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6347d-9a20-4cab-b0ba-3a35e0cf27bb None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:32.085468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e6347d-9a20-4cab-b0ba-3a35e0cf27bb None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:32.146452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:32.181014 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43f727-b341-45c3-8a55-e6ca34cf4900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:32.181639 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43f727-b341-45c3-8a55-e6ca34cf4900 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:32.181935 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43f727-b341-45c3-8a55-e6ca34cf4900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:32.182198 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d43f727-b341-45c3-8a55-e6ca34cf4900 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:32.182811 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d43f727-b341-45c3-8a55-e6ca34cf4900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:32.184513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: WARNING py.warnings [None req-2d43f727-b341-45c3-8a55-e6ca34cf4900 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:08:32.184513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: warnings.warn(msg) Aug 31 19:08:32.184513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: Aug 31 19:08:32.185714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d43f727-b341-45c3-8a55-e6ca34cf4900 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:32.242186 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 747/1497] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:08:32 2022] POST /identity/v3/projects => generated 379 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:08:32.246396 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fd4e1-7c22-47e5-915f-30a0bca2f2cd admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:32.247928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b11fd4e1-7c22-47e5-915f-30a0bca2f2cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:32.280418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11fd4e1-7c22-47e5-915f-30a0bca2f2cd admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:32.371098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baadac98-88c8-4392-9338-e822b8613c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:32.371689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadac98-88c8-4392-9338-e822b8613c08 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:32.371960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadac98-88c8-4392-9338-e822b8613c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:32.372223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baadac98-88c8-4392-9338-e822b8613c08 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:32.372842 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baadac98-88c8-4392-9338-e822b8613c08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:32.373865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: WARNING py.warnings [None req-baadac98-88c8-4392-9338-e822b8613c08 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:08:32.373865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: warnings.warn(msg) Aug 31 19:08:32.373865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: Aug 31 19:08:32.375051 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baadac98-88c8-4392-9338-e822b8613c08 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:32.415932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 751/1498] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:08:32 2022] POST /identity/v3/users => generated 290 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:08:32.420105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43f727-b341-45c3-8a55-e6ca34cf4900 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:32.421627 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2d43f727-b341-45c3-8a55-e6ca34cf4900 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:32.482967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d43f727-b341-45c3-8a55-e6ca34cf4900 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:32.566565 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:32.577890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f41a1-5d39-43a6-988b-949d9b07cf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:32.578488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f41a1-5d39-43a6-988b-949d9b07cf62 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:32.578760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f41a1-5d39-43a6-988b-949d9b07cf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:32.579022 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f41a1-5d39-43a6-988b-949d9b07cf62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:32.579647 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f41a1-5d39-43a6-988b-949d9b07cf62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:32.581135 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: WARNING py.warnings [None req-317f41a1-5d39-43a6-988b-949d9b07cf62 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:08:32.581135 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: warnings.warn(msg) Aug 31 19:08:32.581135 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: Aug 31 19:08:32.582028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317f41a1-5d39-43a6-988b-949d9b07cf62 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:32.594434 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 748/1499] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:08:32.599190 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baadac98-88c8-4392-9338-e822b8613c08 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:32.600679 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-baadac98-88c8-4392-9338-e822b8613c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:32.611862 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baadac98-88c8-4392-9338-e822b8613c08 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:32.636037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:32.705589 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ba944-d788-4be0-97d7-0e740ef69689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:32.706139 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ba944-d788-4be0-97d7-0e740ef69689 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:32.706414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ba944-d788-4be0-97d7-0e740ef69689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:32.706683 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ba944-d788-4be0-97d7-0e740ef69689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:32.707339 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ba944-d788-4be0-97d7-0e740ef69689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:32.708345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: WARNING py.warnings [None req-359ba944-d788-4be0-97d7-0e740ef69689 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:08:32.708345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: warnings.warn(msg) Aug 31 19:08:32.708345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: Aug 31 19:08:32.709246 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ba944-d788-4be0-97d7-0e740ef69689 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:32.721995 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 752/1500] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:08:32.726122 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f41a1-5d39-43a6-988b-949d9b07cf62 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:32.727633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-317f41a1-5d39-43a6-988b-949d9b07cf62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:32.760336 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f41a1-5d39-43a6-988b-949d9b07cf62 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:32.854391 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd90d28-0d70-4ee9-bcca-e715e2c195cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:32.855073 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd90d28-0d70-4ee9-bcca-e715e2c195cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:32.855285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd90d28-0d70-4ee9-bcca-e715e2c195cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:32.855545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd90d28-0d70-4ee9-bcca-e715e2c195cd admin admin] PATH_INFO: `/v3/projects/410fd150cf8e41f0ba1c0075137310a6/users/0ff5b1cd36f343c5827105f00b1cbf5e/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:32.876858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd90d28-0d70-4ee9-bcca-e715e2c195cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=410fd150cf8e41f0ba1c0075137310a6, user_id=0ff5b1cd36f343c5827105f00b1cbf5e, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:32.877938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: WARNING py.warnings [None req-acd90d28-0d70-4ee9-bcca-e715e2c195cd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:08:32.877938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: warnings.warn(msg) Aug 31 19:08:32.877938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: Aug 31 19:08:32.879835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd90d28-0d70-4ee9-bcca-e715e2c195cd admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:32.903862 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 749/1501] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:08:32 2022] PUT /identity/v3/projects/410fd150cf8e41f0ba1c0075137310a6/users/0ff5b1cd36f343c5827105f00b1cbf5e/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:08:32.916447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990467e-b20b-40de-ba3f-b1ca9ce769d5 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:32.916728 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990467e-b20b-40de-ba3f-b1ca9ce769d5 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:32.917000 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e990467e-b20b-40de-ba3f-b1ca9ce769d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:32.973336 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e990467e-b20b-40de-ba3f-b1ca9ce769d5 None None] MFA Rules not processed for user `0ff5b1cd36f343c5827105f00b1cbf5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:08:32.974752 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e990467e-b20b-40de-ba3f-b1ca9ce769d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:33.051015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 753/1502] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:08:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:08:33.064259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399222cf-43bb-4be4-9bdb-a8479215587a None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:33.065061 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399222cf-43bb-4be4-9bdb-a8479215587a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:33.065370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399222cf-43bb-4be4-9bdb-a8479215587a None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:33.066848 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 750/1503] 10.222.0.65 () {54 vars in 964 bytes} [Wed Aug 31 19:08:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:08:33.071663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a5146-bd60-4df0-82d3-8710d346460d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:33.071943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a5146-bd60-4df0-82d3-8710d346460d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:33.072213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a5146-bd60-4df0-82d3-8710d346460d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:33.104007 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:33.148622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-df1a5146-bd60-4df0-82d3-8710d346460d None None] MFA Rules not processed for user `5d157abcf94745a5b92d6df64d872f73`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:08:33.149841 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-df1a5146-bd60-4df0-82d3-8710d346460d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:33.224168 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 754/1504] 10.222.0.65 () {58 vars in 1061 bytes} [Wed Aug 31 19:08:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:08:33.233872 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399222cf-43bb-4be4-9bdb-a8479215587a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:33.235439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-399222cf-43bb-4be4-9bdb-a8479215587a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:33.286257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:33.341147 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399222cf-43bb-4be4-9bdb-a8479215587a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:33.413385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c186ec-ef9d-43d6-aaf5-17b6346dc754 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:33.413986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c186ec-ef9d-43d6-aaf5-17b6346dc754 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:33.414268 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c186ec-ef9d-43d6-aaf5-17b6346dc754 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:33.414540 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c186ec-ef9d-43d6-aaf5-17b6346dc754 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:33.416129 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-66c186ec-ef9d-43d6-aaf5-17b6346dc754 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:33.513283 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c186ec-ef9d-43d6-aaf5-17b6346dc754 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:33.514695 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c186ec-ef9d-43d6-aaf5-17b6346dc754 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:33.518128 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:33.532533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 751/1505] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:08:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:08:33.577346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a95695-b2ba-4280-b0f0-c88da81b6518 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:33.577632 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a95695-b2ba-4280-b0f0-c88da81b6518 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:33.577905 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a95695-b2ba-4280-b0f0-c88da81b6518 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:33.579155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 755/1506] 10.222.0.65 () {52 vars in 830 bytes} [Wed Aug 31 19:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:08:33.582701 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c186ec-ef9d-43d6-aaf5-17b6346dc754 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:33.584003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-66c186ec-ef9d-43d6-aaf5-17b6346dc754 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:33.681408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c186ec-ef9d-43d6-aaf5-17b6346dc754 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:33.773839 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb0631-25dc-46b2-a19c-e71a33e5db89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:33.774416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb0631-25dc-46b2-a19c-e71a33e5db89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:33.774696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb0631-25dc-46b2-a19c-e71a33e5db89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:33.774969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bb0631-25dc-46b2-a19c-e71a33e5db89 admin admin] PATH_INFO: `/v3/users/0ff5b1cd36f343c5827105f00b1cbf5e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:33.787524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb0631-25dc-46b2-a19c-e71a33e5db89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ff5b1cd36f343c5827105f00b1cbf5e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:33.788539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: WARNING py.warnings [None req-76bb0631-25dc-46b2-a19c-e71a33e5db89 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:08:33.788539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: warnings.warn(msg) Aug 31 19:08:33.788539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: Aug 31 19:08:33.789808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bb0631-25dc-46b2-a19c-e71a33e5db89 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:33.800557 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:33.858047 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-76bb0631-25dc-46b2-a19c-e71a33e5db89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ff5b1cd36f343c5827105f00b1cbf5e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:08:33.872965 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-76bb0631-25dc-46b2-a19c-e71a33e5db89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ff5b1cd36f343c5827105f00b1cbf5e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:08:33.881920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-76bb0631-25dc-46b2-a19c-e71a33e5db89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ff5b1cd36f343c5827105f00b1cbf5e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:08:33.882385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-76bb0631-25dc-46b2-a19c-e71a33e5db89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ff5b1cd36f343c5827105f00b1cbf5e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:08:33.904761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-76bb0631-25dc-46b2-a19c-e71a33e5db89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ff5b1cd36f343c5827105f00b1cbf5e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:08:33.910807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 752/1507] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:08:33 2022] DELETE /identity/v3/users/0ff5b1cd36f343c5827105f00b1cbf5e => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:08:33.915186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b35e5-1dbe-4edd-ba1b-9c4a82b4f77f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:33.915674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b35e5-1dbe-4edd-ba1b-9c4a82b4f77f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:33.915784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b35e5-1dbe-4edd-ba1b-9c4a82b4f77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:33.971378 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f02b35e5-1dbe-4edd-ba1b-9c4a82b4f77f None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:08:33.972643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f02b35e5-1dbe-4edd-ba1b-9c4a82b4f77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:34.046041 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:34.058279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 756/1508] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:08:34.068379 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb0631-25dc-46b2-a19c-e71a33e5db89 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:34.069651 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-76bb0631-25dc-46b2-a19c-e71a33e5db89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:34.177486 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bb0631-25dc-46b2-a19c-e71a33e5db89 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:34.233033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:34.253329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3ba2bb-bd74-433f-a610-24cfda53a90f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:34.253940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ba2bb-bd74-433f-a610-24cfda53a90f service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:34.254214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ba2bb-bd74-433f-a610-24cfda53a90f service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:34.254488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3ba2bb-bd74-433f-a610-24cfda53a90f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:34.256094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1c3ba2bb-bd74-433f-a610-24cfda53a90f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:34.362287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ba2bb-bd74-433f-a610-24cfda53a90f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:34.363642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3ba2bb-bd74-433f-a610-24cfda53a90f service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:34.381811 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 753/1509] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:08:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:08:34.713947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b35e5-1dbe-4edd-ba1b-9c4a82b4f77f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:08:34.715307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f02b35e5-1dbe-4edd-ba1b-9c4a82b4f77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:08:34.726818 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b35e5-1dbe-4edd-ba1b-9c4a82b4f77f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:08:34.824480 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dac78d-403a-42b0-9ad6-62d04a0d485b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:08:34.825116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dac78d-403a-42b0-9ad6-62d04a0d485b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:08:34.825406 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dac78d-403a-42b0-9ad6-62d04a0d485b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:08:34.825676 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dac78d-403a-42b0-9ad6-62d04a0d485b admin admin] PATH_INFO: `/v3/projects/410fd150cf8e41f0ba1c0075137310a6` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:08:34.835477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dac78d-403a-42b0-9ad6-62d04a0d485b admin admin] RBAC: Authorizing `identity:delete_project(project_id=410fd150cf8e41f0ba1c0075137310a6)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:08:34.836523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: WARNING py.warnings [None req-f5dac78d-403a-42b0-9ad6-62d04a0d485b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:08:34.836523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: warnings.warn(msg) Aug 31 19:08:34.836523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: Aug 31 19:08:34.837740 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dac78d-403a-42b0-9ad6-62d04a0d485b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:08:34.894205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f5dac78d-403a-42b0-9ad6-62d04a0d485b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '410fd150cf8e41f0ba1c0075137310a6', 'request_id': 'req-f5dac78d-403a-42b0-9ad6-62d04a0d485b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:08:34.905514 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f5dac78d-403a-42b0-9ad6-62d04a0d485b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '410fd150cf8e41f0ba1c0075137310a6', 'request_id': 'req-f5dac78d-403a-42b0-9ad6-62d04a0d485b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:08:34.911574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f5dac78d-403a-42b0-9ad6-62d04a0d485b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '410fd150cf8e41f0ba1c0075137310a6', 'request_id': 'req-f5dac78d-403a-42b0-9ad6-62d04a0d485b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:08:34.941508 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f5dac78d-403a-42b0-9ad6-62d04a0d485b admin admin] The token cache is being invalidate because project 410fd150cf8e41f0ba1c0075137310a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:08:34.942125 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f5dac78d-403a-42b0-9ad6-62d04a0d485b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:08:34.942297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f5dac78d-403a-42b0-9ad6-62d04a0d485b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:08:34.943586 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 757/1510] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:08:34 2022] DELETE /identity/v3/projects/410fd150cf8e41f0ba1c0075137310a6 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:08:44.379544 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=62 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:08:44.938896 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:08.031235 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7175476-128b-4e33-9d95-f7c0f4914c33 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:08.031871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7175476-128b-4e33-9d95-f7c0f4914c33 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:08.031871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7175476-128b-4e33-9d95-f7c0f4914c33 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:08.033126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:08.038699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a17058-598f-4c46-ae48-3075fd6c7fe1 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:08.039141 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a17058-598f-4c46-ae48-3075fd6c7fe1 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:08.039236 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a17058-598f-4c46-ae48-3075fd6c7fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:08.120429 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a1a17058-598f-4c46-ae48-3075fd6c7fe1 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:08.121663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a1a17058-598f-4c46-ae48-3075fd6c7fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:08.222549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:08.376704 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7175476-128b-4e33-9d95-f7c0f4914c33 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:08.378951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b7175476-128b-4e33-9d95-f7c0f4914c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:08.499980 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7175476-128b-4e33-9d95-f7c0f4914c33 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:08.505415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aaa46f0-b62f-4b3f-a807-566705803527 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:08.506018 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa46f0-b62f-4b3f-a807-566705803527 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:08.506297 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa46f0-b62f-4b3f-a807-566705803527 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:08.506567 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aaa46f0-b62f-4b3f-a807-566705803527 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:08.514572 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa46f0-b62f-4b3f-a807-566705803527 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:08.516133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aaa46f0-b62f-4b3f-a807-566705803527 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:08.526112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 755/1513] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:09.346234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf83d6f-2cd1-4906-8ff4-6cd29bb94283 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:09.346916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf83d6f-2cd1-4906-8ff4-6cd29bb94283 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:09.346916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf83d6f-2cd1-4906-8ff4-6cd29bb94283 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:09.348144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:09.352926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ad441-467c-4212-9738-459ed807c9e6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:09.353355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ad441-467c-4212-9738-459ed807c9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:09.353512 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502ad441-467c-4212-9738-459ed807c9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:09.373699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-502ad441-467c-4212-9738-459ed807c9e6 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:09.374915 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-502ad441-467c-4212-9738-459ed807c9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:09.383378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:09.537902 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf83d6f-2cd1-4906-8ff4-6cd29bb94283 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:09.552513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf83d6f-2cd1-4906-8ff4-6cd29bb94283 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:09.557813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9639624b-a3e0-4ed8-ba7f-e14d3670e3d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:09.558395 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639624b-a3e0-4ed8-ba7f-e14d3670e3d2 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:09.558666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639624b-a3e0-4ed8-ba7f-e14d3670e3d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:09.558928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9639624b-a3e0-4ed8-ba7f-e14d3670e3d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:09.566745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9639624b-a3e0-4ed8-ba7f-e14d3670e3d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:09.568298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9639624b-a3e0-4ed8-ba7f-e14d3670e3d2 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:09.577497 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 760/1516] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:09.925281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf8b9c-022a-4e73-9f66-29caba6f462a None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:09.925507 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf8b9c-022a-4e73-9f66-29caba6f462a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:09.925821 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf8b9c-022a-4e73-9f66-29caba6f462a None None] PATH_INFO: `/v3` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:09.927097 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 757/1517] 10.222.0.94 () {54 vars in 925 bytes} [Wed Aug 31 19:10:09 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:10:09.933379 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a4378-a7c8-46da-919b-b00f99c767f6 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:09.933666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a4378-a7c8-46da-919b-b00f99c767f6 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:09.933940 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a4378-a7c8-46da-919b-b00f99c767f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:09.992604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-7b8a4378-a7c8-46da-919b-b00f99c767f6 None None] MFA Rules not processed for user `74f1846e395e4d5483fcbb41eb28a77d`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:09.993830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7b8a4378-a7c8-46da-919b-b00f99c767f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:10.010063 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:10.064860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 761/1518] 10.222.0.94 () {58 vars in 1015 bytes} [Wed Aug 31 19:10:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:19.227786 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c5639-e801-4565-bef7-24de5b2523a2 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:19.228294 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c5639-e801-4565-bef7-24de5b2523a2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:19.228410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733c5639-e801-4565-bef7-24de5b2523a2 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:19.229763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 758/1519] 10.222.0.65 () {54 vars in 961 bytes} [Wed Aug 31 19:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:19.234580 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28196881-695d-42cc-bf0e-c25f75192687 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:19.235039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28196881-695d-42cc-bf0e-c25f75192687 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:19.235144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28196881-695d-42cc-bf0e-c25f75192687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:19.254658 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-28196881-695d-42cc-bf0e-c25f75192687 None None] MFA Rules not processed for user `74f1846e395e4d5483fcbb41eb28a77d`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:19.255836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-28196881-695d-42cc-bf0e-c25f75192687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:19.264183 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 762/1520] 10.222.0.65 () {58 vars in 1058 bytes} [Wed Aug 31 19:10:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:10:19.272387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c5639-e801-4565-bef7-24de5b2523a2 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:19.285429 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733c5639-e801-4565-bef7-24de5b2523a2 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:19.290268 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6195b744-b152-4276-9f81-753aeecdf337 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:19.290844 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195b744-b152-4276-9f81-753aeecdf337 service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:19.291118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195b744-b152-4276-9f81-753aeecdf337 service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:19.291385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6195b744-b152-4276-9f81-753aeecdf337 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:19.299296 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195b744-b152-4276-9f81-753aeecdf337 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:19.300893 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6195b744-b152-4276-9f81-753aeecdf337 service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:19.310173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 759/1521] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:20.794556 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199d920-3c52-4486-b022-c08921e34b3f None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:20.795128 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199d920-3c52-4486-b022-c08921e34b3f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:20.795128 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a199d920-3c52-4486-b022-c08921e34b3f None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:20.796475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:20.801912 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593605c-6356-4ffe-9fb1-fa37874eceee None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:20.802364 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593605c-6356-4ffe-9fb1-fa37874eceee None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:20.802469 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4593605c-6356-4ffe-9fb1-fa37874eceee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:20.821489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4593605c-6356-4ffe-9fb1-fa37874eceee None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:20.822674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4593605c-6356-4ffe-9fb1-fa37874eceee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:20.830489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:10:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:20.985285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199d920-3c52-4486-b022-c08921e34b3f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:21.000641 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a199d920-3c52-4486-b022-c08921e34b3f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:21.006203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eba9c9-586c-4e20-9ff0-08086581a49d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:21.006800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eba9c9-586c-4e20-9ff0-08086581a49d service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:21.007076 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eba9c9-586c-4e20-9ff0-08086581a49d service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:21.007348 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eba9c9-586c-4e20-9ff0-08086581a49d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:21.015337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eba9c9-586c-4e20-9ff0-08086581a49d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:21.016903 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eba9c9-586c-4e20-9ff0-08086581a49d service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:21.026187 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 764/1524] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:21.845588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31487b-8ac7-415a-965e-92f18fa70409 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:21.846171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31487b-8ac7-415a-965e-92f18fa70409 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:21.846171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c31487b-8ac7-415a-965e-92f18fa70409 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:21.847473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 31 19:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:21.852219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb58389-d7c9-4144-9711-9c39c33d4b68 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:21.852693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb58389-d7c9-4144-9711-9c39c33d4b68 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:21.852836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb58389-d7c9-4144-9711-9c39c33d4b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:21.873998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3bb58389-d7c9-4144-9711-9c39c33d4b68 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:21.875188 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3bb58389-d7c9-4144-9711-9c39c33d4b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:21.883785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 31 19:10:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:22.040921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31487b-8ac7-415a-965e-92f18fa70409 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:22.055699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c31487b-8ac7-415a-965e-92f18fa70409 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:22.061038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cb8291-8972-4538-90e3-e9ca7739c13b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:22.061654 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb8291-8972-4538-90e3-e9ca7739c13b service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:22.061933 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb8291-8972-4538-90e3-e9ca7739c13b service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:22.062206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cb8291-8972-4538-90e3-e9ca7739c13b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:22.070702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb8291-8972-4538-90e3-e9ca7739c13b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:22.072264 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cb8291-8972-4538-90e3-e9ca7739c13b service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:22.081589 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 762/1527] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:25.993930 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915f9c9-37d4-4a5c-a9bc-60b3e71185f0 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:25.994498 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915f9c9-37d4-4a5c-a9bc-60b3e71185f0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:25.994498 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915f9c9-37d4-4a5c-a9bc-60b3e71185f0 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:25.995871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:26.001271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266cbf79-403f-43e1-bd5f-90d41c1b328b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:26.001691 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266cbf79-403f-43e1-bd5f-90d41c1b328b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:26.001880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266cbf79-403f-43e1-bd5f-90d41c1b328b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:26.021762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-266cbf79-403f-43e1-bd5f-90d41c1b328b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:26.022956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-266cbf79-403f-43e1-bd5f-90d41c1b328b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:26.030937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:26.319140 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915f9c9-37d4-4a5c-a9bc-60b3e71185f0 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:26.320806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c915f9c9-37d4-4a5c-a9bc-60b3e71185f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:26.398105 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915f9c9-37d4-4a5c-a9bc-60b3e71185f0 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:26.401545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393afe17-d590-4ce7-8f1a-858073e67104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:26.402135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393afe17-d590-4ce7-8f1a-858073e67104 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:26.402416 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393afe17-d590-4ce7-8f1a-858073e67104 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:26.402687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393afe17-d590-4ce7-8f1a-858073e67104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:26.412021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393afe17-d590-4ce7-8f1a-858073e67104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:26.413498 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393afe17-d590-4ce7-8f1a-858073e67104 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:26.422724 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 767/1530] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:28.338976 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba59aff-ebfb-4838-a5a2-0deb4f458f2d None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:28.339566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba59aff-ebfb-4838-a5a2-0deb4f458f2d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:28.339566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba59aff-ebfb-4838-a5a2-0deb4f458f2d None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:28.340875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:28.346002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c018c-f71c-4a74-9dd3-9741e09876ff None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:28.346432 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c018c-f71c-4a74-9dd3-9741e09876ff None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:28.346658 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c018c-f71c-4a74-9dd3-9741e09876ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:28.367035 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2a6c018c-f71c-4a74-9dd3-9741e09876ff None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:28.368299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2a6c018c-f71c-4a74-9dd3-9741e09876ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:28.376291 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:10:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:28.661613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba59aff-ebfb-4838-a5a2-0deb4f458f2d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:28.663320 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9ba59aff-ebfb-4838-a5a2-0deb4f458f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:28.676375 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba59aff-ebfb-4838-a5a2-0deb4f458f2d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:28.679991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cf2076-6f44-475f-91fc-b308c29b731c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:28.680579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf2076-6f44-475f-91fc-b308c29b731c service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:28.680856 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf2076-6f44-475f-91fc-b308c29b731c service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:28.681125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf2076-6f44-475f-91fc-b308c29b731c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:28.689875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf2076-6f44-475f-91fc-b308c29b731c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:28.691286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4cf2076-6f44-475f-91fc-b308c29b731c service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:28.700658 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 765/1533] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:32.121282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66fd31-a045-4353-82f7-4d9c63c5f3b8 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:32.121849 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66fd31-a045-4353-82f7-4d9c63c5f3b8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:32.121849 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66fd31-a045-4353-82f7-4d9c63c5f3b8 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:32.123182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:32.128658 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23d7b2-6012-45eb-9a29-64e2d760b148 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:32.129090 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23d7b2-6012-45eb-9a29-64e2d760b148 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:32.129214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf23d7b2-6012-45eb-9a29-64e2d760b148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:32.148116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-cf23d7b2-6012-45eb-9a29-64e2d760b148 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:32.149334 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cf23d7b2-6012-45eb-9a29-64e2d760b148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:32.156984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:10:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:32.721238 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66fd31-a045-4353-82f7-4d9c63c5f3b8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:32.722902 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-aa66fd31-a045-4353-82f7-4d9c63c5f3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:32.786599 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:32.799624 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66fd31-a045-4353-82f7-4d9c63c5f3b8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:32.803112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1ae47-e197-4ffe-9719-ff2e2a786318 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:32.803699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1ae47-e197-4ffe-9719-ff2e2a786318 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:32.803970 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1ae47-e197-4ffe-9719-ff2e2a786318 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:32.804229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1ae47-e197-4ffe-9719-ff2e2a786318 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:32.812773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1ae47-e197-4ffe-9719-ff2e2a786318 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:32.814203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f1ae47-e197-4ffe-9719-ff2e2a786318 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:32.823782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 770/1536] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:34.740071 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733df6b-d39b-41b6-a471-bb5b32404e5e None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:34.740624 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733df6b-d39b-41b6-a471-bb5b32404e5e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:34.740624 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733df6b-d39b-41b6-a471-bb5b32404e5e None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:34.741988 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 31 19:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:10:34.747139 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde72b23-f2fe-4c73-b705-6af05bae83b5 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:34.747555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde72b23-f2fe-4c73-b705-6af05bae83b5 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:34.747685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde72b23-f2fe-4c73-b705-6af05bae83b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:34.766864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-fde72b23-f2fe-4c73-b705-6af05bae83b5 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:10:34.768043 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-fde72b23-f2fe-4c73-b705-6af05bae83b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:34.776236 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 31 19:10:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:10:35.330645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733df6b-d39b-41b6-a471-bb5b32404e5e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:10:35.332551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4733df6b-d39b-41b6-a471-bb5b32404e5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:10:35.340697 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=84 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:35.347146 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733df6b-d39b-41b6-a471-bb5b32404e5e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:10:35.350643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542184fb-e3e4-47f8-b3f9-cc686e133b3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:10:35.351228 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542184fb-e3e4-47f8-b3f9-cc686e133b3b service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:10:35.351584 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542184fb-e3e4-47f8-b3f9-cc686e133b3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:10:35.351880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542184fb-e3e4-47f8-b3f9-cc686e133b3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:10:35.360672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542184fb-e3e4-47f8-b3f9-cc686e133b3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:10:35.362279 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542184fb-e3e4-47f8-b3f9-cc686e133b3b service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:10:35.372191 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 768/1539] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:10:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:10:44.762087 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=11 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:10:45.368224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:01.625333 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9e33a-56fd-4374-bf35-8331d3a1c8d6 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:01.625950 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9e33a-56fd-4374-bf35-8331d3a1c8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:01.625950 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d9e33a-56fd-4374-bf35-8331d3a1c8d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:01.650941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f2d9e33a-56fd-4374-bf35-8331d3a1c8d6 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:01.652731 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f2d9e33a-56fd-4374-bf35-8331d3a1c8d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:01.668814 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 772/1540] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:01.678416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80327ab-a0ea-4864-aa23-9159b1207441 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:01.679033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80327ab-a0ea-4864-aa23-9159b1207441 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:01.679033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80327ab-a0ea-4864-aa23-9159b1207441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:01.707200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a80327ab-a0ea-4864-aa23-9159b1207441 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:01.708608 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a80327ab-a0ea-4864-aa23-9159b1207441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:01.721906 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 769/1541] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:01.726785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9e33a-56fd-4374-bf35-8331d3a1c8d6 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:01.730423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b93168-2bcc-444c-81e5-cc8b0c65644c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:01.730727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b93168-2bcc-444c-81e5-cc8b0c65644c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:01.730968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b93168-2bcc-444c-81e5-cc8b0c65644c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:01.743274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d9e33a-56fd-4374-bf35-8331d3a1c8d6 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:01.750506 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80d0728-7b18-4070-a7e5-d1efbf447b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:01.751145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d0728-7b18-4070-a7e5-d1efbf447b95 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:01.751425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d0728-7b18-4070-a7e5-d1efbf447b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:01.751698 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80d0728-7b18-4070-a7e5-d1efbf447b95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:01.753108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d0728-7b18-4070-a7e5-d1efbf447b95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:01.753443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d0728-7b18-4070-a7e5-d1efbf447b95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:01.755095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80d0728-7b18-4070-a7e5-d1efbf447b95 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:01.770265 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-97b93168-2bcc-444c-81e5-cc8b0c65644c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:01.772091 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-97b93168-2bcc-444c-81e5-cc8b0c65644c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:01.779611 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 773/1542] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:12:01.786005 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 770/1543] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:12:01.795155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dd0a0-bd03-490f-bcc6-e4eaadd6aa0c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:01.795415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b93168-2bcc-444c-81e5-cc8b0c65644c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:01.795529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dd0a0-bd03-490f-bcc6-e4eaadd6aa0c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:01.795762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dd0a0-bd03-490f-bcc6-e4eaadd6aa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:01.817083 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b93168-2bcc-444c-81e5-cc8b0c65644c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:01.827437 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e39dd0a0-bd03-490f-bcc6-e4eaadd6aa0c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:01.828664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e39dd0a0-bd03-490f-bcc6-e4eaadd6aa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:01.829295 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a517feb-987c-43ed-a784-2b0e8cfa764e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:01.829912 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a517feb-987c-43ed-a784-2b0e8cfa764e admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:01.830191 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a517feb-987c-43ed-a784-2b0e8cfa764e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:01.830452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a517feb-987c-43ed-a784-2b0e8cfa764e admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:01.831195 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a517feb-987c-43ed-a784-2b0e8cfa764e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:01.833274 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a517feb-987c-43ed-a784-2b0e8cfa764e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:01.843894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 774/1544] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:01.848714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39dd0a0-bd03-490f-bcc6-e4eaadd6aa0c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:01.869206 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39dd0a0-bd03-490f-bcc6-e4eaadd6aa0c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:01.888335 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4430e-9bed-41c6-812b-88b7d6c1ddce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:01.889020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4430e-9bed-41c6-812b-88b7d6c1ddce admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:01.889499 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4430e-9bed-41c6-812b-88b7d6c1ddce admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:01.889782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4430e-9bed-41c6-812b-88b7d6c1ddce admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:01.890426 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4430e-9bed-41c6-812b-88b7d6c1ddce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:01.890806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4430e-9bed-41c6-812b-88b7d6c1ddce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:01.892843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab4430e-9bed-41c6-812b-88b7d6c1ddce admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:01.912026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 771/1545] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:01 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:12:01.917342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a517feb-987c-43ed-a784-2b0e8cfa764e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:01.917675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 775/1546] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:12:01.925775 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4430e-9bed-41c6-812b-88b7d6c1ddce admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:01.945212 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4430e-9bed-41c6-812b-88b7d6c1ddce admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.031860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a517feb-987c-43ed-a784-2b0e8cfa764e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.042495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f675d4-3afd-4d1f-afdc-8abbf27d218c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.043118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f675d4-3afd-4d1f-afdc-8abbf27d218c admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.043394 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f675d4-3afd-4d1f-afdc-8abbf27d218c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.043665 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f675d4-3afd-4d1f-afdc-8abbf27d218c admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.044300 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f675d4-3afd-4d1f-afdc-8abbf27d218c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.045342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: WARNING py.warnings [None req-59f675d4-3afd-4d1f-afdc-8abbf27d218c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:12:02.045342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: warnings.warn(msg) Aug 31 19:12:02.045342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: Aug 31 19:12:02.046652 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f675d4-3afd-4d1f-afdc-8abbf27d218c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.053235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f14f33-c537-4fad-8880-999dcbccf7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.053860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f14f33-c537-4fad-8880-999dcbccf7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.054137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f14f33-c537-4fad-8880-999dcbccf7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.054405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f14f33-c537-4fad-8880-999dcbccf7c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.055041 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f14f33-c537-4fad-8880-999dcbccf7c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.057858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: WARNING py.warnings [None req-44f14f33-c537-4fad-8880-999dcbccf7c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:12:02.057858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: warnings.warn(msg) Aug 31 19:12:02.057858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: Aug 31 19:12:02.059153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f14f33-c537-4fad-8880-999dcbccf7c2 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.071404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 772/1547] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:01 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:12:02.072804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd7aec-7214-4c80-97e5-e363d06853b6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.073688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd7aec-7214-4c80-97e5-e363d06853b6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.074011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dd7aec-7214-4c80-97e5-e363d06853b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.091421 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 776/1548] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:01 2022] POST /identity/v3/projects => generated 373 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:12:02.092666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68528ff-1424-4593-871b-9ee4aba21637 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.092946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68528ff-1424-4593-871b-9ee4aba21637 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.093257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68528ff-1424-4593-871b-9ee4aba21637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.097337 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d9dd7aec-7214-4c80-97e5-e363d06853b6 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:02.098574 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d9dd7aec-7214-4c80-97e5-e363d06853b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:02.112874 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e68528ff-1424-4593-871b-9ee4aba21637 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:02.114089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e68528ff-1424-4593-871b-9ee4aba21637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:02.159338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 773/1549] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:02.160493 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffabbb-0a41-4d61-9253-5ae484997d3b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.160773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffabbb-0a41-4d61-9253-5ae484997d3b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.161043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffabbb-0a41-4d61-9253-5ae484997d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.187887 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 777/1550] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:02.188118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4dffabbb-0a41-4d61-9253-5ae484997d3b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:02.188225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf471b2-c6f4-4e2a-8713-d6c5c6a840b0 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.188225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf471b2-c6f4-4e2a-8713-d6c5c6a840b0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.188225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf471b2-c6f4-4e2a-8713-d6c5c6a840b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.188472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4dffabbb-0a41-4d61-9253-5ae484997d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:02.191669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 774/1551] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:02.194445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12764e2e-4769-4eca-b550-9ac5c0709fb9 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.194732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12764e2e-4769-4eca-b550-9ac5c0709fb9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.195006 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12764e2e-4769-4eca-b550-9ac5c0709fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.203599 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2bf471b2-c6f4-4e2a-8713-d6c5c6a840b0 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:02.204645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2bf471b2-c6f4-4e2a-8713-d6c5c6a840b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:02.212531 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 778/1552] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:02.213344 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf471b2-c6f4-4e2a-8713-d6c5c6a840b0 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.215985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-12764e2e-4769-4eca-b550-9ac5c0709fb9 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:02.217172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-12764e2e-4769-4eca-b550-9ac5c0709fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:02.225172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 775/1553] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:02.226392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f95df-3ce5-4c7c-829c-8daa6653cdce None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.226671 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f95df-3ce5-4c7c-829c-8daa6653cdce None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.226937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f95df-3ce5-4c7c-829c-8daa6653cdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.230398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf471b2-c6f4-4e2a-8713-d6c5c6a840b0 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.235034 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c537ef8-870c-4ede-bd28-fb5c1319f33f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.235882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c537ef8-870c-4ede-bd28-fb5c1319f33f admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.236160 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c537ef8-870c-4ede-bd28-fb5c1319f33f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.236424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c537ef8-870c-4ede-bd28-fb5c1319f33f admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.237055 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c537ef8-870c-4ede-bd28-fb5c1319f33f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.238668 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c537ef8-870c-4ede-bd28-fb5c1319f33f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.246871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0c7f95df-3ce5-4c7c-829c-8daa6653cdce None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:02.248063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0c7f95df-3ce5-4c7c-829c-8daa6653cdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:02.252416 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 779/1554] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:02.252416 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c537ef8-870c-4ede-bd28-fb5c1319f33f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.256430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 776/1555] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:02.257668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec8e86-447e-4f09-9f5b-02a4778aac4f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.257940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec8e86-447e-4f09-9f5b-02a4778aac4f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.258215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec8e86-447e-4f09-9f5b-02a4778aac4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.260774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c537ef8-870c-4ede-bd28-fb5c1319f33f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.265519 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c42028-fb41-4f23-a5b5-e820b8fdd5d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.266106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c42028-fb41-4f23-a5b5-e820b8fdd5d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.266381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c42028-fb41-4f23-a5b5-e820b8fdd5d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.266652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c42028-fb41-4f23-a5b5-e820b8fdd5d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.267277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c42028-fb41-4f23-a5b5-e820b8fdd5d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.269179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c42028-fb41-4f23-a5b5-e820b8fdd5d9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.277644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ddec8e86-447e-4f09-9f5b-02a4778aac4f None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:02.278824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ddec8e86-447e-4f09-9f5b-02a4778aac4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:02.286826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 777/1556] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:02.290587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b22f7f-026c-47c6-9861-32a8cecdd210 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.290769 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b22f7f-026c-47c6-9861-32a8cecdd210 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.290940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b22f7f-026c-47c6-9861-32a8cecdd210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.293477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 780/1557] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:02.294966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a21d45-d32a-4c41-8d0c-6161c9319948 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.295302 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a21d45-d32a-4c41-8d0c-6161c9319948 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.295575 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a21d45-d32a-4c41-8d0c-6161c9319948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.311008 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-74b22f7f-026c-47c6-9861-32a8cecdd210 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:02.312352 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-74b22f7f-026c-47c6-9861-32a8cecdd210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:02.315505 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c6a21d45-d32a-4c41-8d0c-6161c9319948 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:02.316677 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c6a21d45-d32a-4c41-8d0c-6161c9319948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:02.320377 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 778/1558] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:02.321662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b3839-47c6-4b5c-a5c5-4822a1292718 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.321941 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b3839-47c6-4b5c-a5c5-4822a1292718 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.322210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b3839-47c6-4b5c-a5c5-4822a1292718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.328291 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 781/1559] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:02.329580 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b32bd0-d796-4da2-853a-c132d535f397 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.329853 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b32bd0-d796-4da2-853a-c132d535f397 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.330132 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b32bd0-d796-4da2-853a-c132d535f397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.338410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:02.342701 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-057b3839-47c6-4b5c-a5c5-4822a1292718 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:02.343880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-057b3839-47c6-4b5c-a5c5-4822a1292718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:02.350815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e6b32bd0-d796-4da2-853a-c132d535f397 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:02.351991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e6b32bd0-d796-4da2-853a-c132d535f397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:02.353023 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 779/1560] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:02.353951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b3839-47c6-4b5c-a5c5-4822a1292718 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.364338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 782/1561] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:02.364608 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b3839-47c6-4b5c-a5c5-4822a1292718 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.365576 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234846f5-6464-442f-816f-46ad1a150bb3 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.365855 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234846f5-6464-442f-816f-46ad1a150bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.366135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234846f5-6464-442f-816f-46ad1a150bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.372029 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc6a96-9079-47fd-8a7d-a7df6fe8f159 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.372617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc6a96-9079-47fd-8a7d-a7df6fe8f159 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.372894 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc6a96-9079-47fd-8a7d-a7df6fe8f159 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.373164 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cc6a96-9079-47fd-8a7d-a7df6fe8f159 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.373825 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc6a96-9079-47fd-8a7d-a7df6fe8f159 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.375772 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cc6a96-9079-47fd-8a7d-a7df6fe8f159 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.377180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:02.387246 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-234846f5-6464-442f-816f-46ad1a150bb3 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:02.387504 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 780/1562] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:02.388458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc6a96-9079-47fd-8a7d-a7df6fe8f159 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.388569 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-234846f5-6464-442f-816f-46ad1a150bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:02.396614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 783/1563] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:02.400165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234846f5-6464-442f-816f-46ad1a150bb3 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.400286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cc6a96-9079-47fd-8a7d-a7df6fe8f159 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.402782 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff06998-2b40-475d-91c8-b110ac262be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.403357 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff06998-2b40-475d-91c8-b110ac262be0 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.403634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff06998-2b40-475d-91c8-b110ac262be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.403905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff06998-2b40-475d-91c8-b110ac262be0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.404531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff06998-2b40-475d-91c8-b110ac262be0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:02.404827 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff06998-2b40-475d-91c8-b110ac262be0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.405865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: WARNING py.warnings [None req-aff06998-2b40-475d-91c8-b110ac262be0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:12:02.405865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: warnings.warn(msg) Aug 31 19:12:02.405865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: Aug 31 19:12:02.406741 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff06998-2b40-475d-91c8-b110ac262be0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.407302 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234846f5-6464-442f-816f-46ad1a150bb3 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.411915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58fee5-57c4-4095-a2f5-a39a16ce2560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.412433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58fee5-57c4-4095-a2f5-a39a16ce2560 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.412709 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58fee5-57c4-4095-a2f5-a39a16ce2560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.412977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58fee5-57c4-4095-a2f5-a39a16ce2560 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.413641 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58fee5-57c4-4095-a2f5-a39a16ce2560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.415204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58fee5-57c4-4095-a2f5-a39a16ce2560 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.419165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 781/1564] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:12:02.419943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff06998-2b40-475d-91c8-b110ac262be0 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.425302 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 784/1565] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:02.426231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58fee5-57c4-4095-a2f5-a39a16ce2560 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.436482 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58fee5-57c4-4095-a2f5-a39a16ce2560 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.437967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff06998-2b40-475d-91c8-b110ac262be0 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.441072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6814bf6-1f69-4065-8697-c420be8c1c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.441673 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6814bf6-1f69-4065-8697-c420be8c1c09 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.441950 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6814bf6-1f69-4065-8697-c420be8c1c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.442218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6814bf6-1f69-4065-8697-c420be8c1c09 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.442569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7b04d-085a-4137-b866-b9f0cc32cd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.442837 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6814bf6-1f69-4065-8697-c420be8c1c09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:02.443162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7b04d-085a-4137-b866-b9f0cc32cd59 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.443310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6814bf6-1f69-4065-8697-c420be8c1c09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.443470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7b04d-085a-4137-b866-b9f0cc32cd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.443711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c7b04d-085a-4137-b866-b9f0cc32cd59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.444338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7b04d-085a-4137-b866-b9f0cc32cd59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:02.444637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7b04d-085a-4137-b866-b9f0cc32cd59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.444746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6814bf6-1f69-4065-8697-c420be8c1c09 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.446229 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c7b04d-085a-4137-b866-b9f0cc32cd59 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.464222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 785/1566] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:12:02.464222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6814bf6-1f69-4065-8697-c420be8c1c09 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.464472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 782/1567] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:12:02.464472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7b04d-085a-4137-b866-b9f0cc32cd59 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.468859 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c7b04d-085a-4137-b866-b9f0cc32cd59 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.470810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6814bf6-1f69-4065-8697-c420be8c1c09 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.473546 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd43ca-5157-4746-9f18-7388b9f1d413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.474134 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd43ca-5157-4746-9f18-7388b9f1d413 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.474412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd43ca-5157-4746-9f18-7388b9f1d413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.474682 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd43ca-5157-4746-9f18-7388b9f1d413 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.475315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd43ca-5157-4746-9f18-7388b9f1d413 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:02.475613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd43ca-5157-4746-9f18-7388b9f1d413 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.476253 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d5bb3-fba4-4459-b1e0-ffdb6974856b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.476936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d5bb3-fba4-4459-b1e0-ffdb6974856b admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.477174 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebd43ca-5157-4746-9f18-7388b9f1d413 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.477522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d5bb3-fba4-4459-b1e0-ffdb6974856b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.477892 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d5bb3-fba4-4459-b1e0-ffdb6974856b admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.478822 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d5bb3-fba4-4459-b1e0-ffdb6974856b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:02.479126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d5bb3-fba4-4459-b1e0-ffdb6974856b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.481334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d5bb3-fba4-4459-b1e0-ffdb6974856b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.489602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 783/1568] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:12:02.490370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd43ca-5157-4746-9f18-7388b9f1d413 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.498275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 786/1569] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:12:02.498938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d5bb3-fba4-4459-b1e0-ffdb6974856b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.507482 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebd43ca-5157-4746-9f18-7388b9f1d413 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.508635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d5bb3-fba4-4459-b1e0-ffdb6974856b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.512590 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3f005-e5c5-4394-9599-61bd3c849346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.513261 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3f005-e5c5-4394-9599-61bd3c849346 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.513539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3f005-e5c5-4394-9599-61bd3c849346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.513649 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cb2f6-5bd3-4acc-8f50-e8f531812280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.513811 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b3f005-e5c5-4394-9599-61bd3c849346 admin admin] PATH_INFO: `/v3/projects/1b4974b4192c453e8824af0c0b2aaaab/users/b9b0860b541a43278543b82b9e775bbb/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.514207 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cb2f6-5bd3-4acc-8f50-e8f531812280 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.514482 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cb2f6-5bd3-4acc-8f50-e8f531812280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.514748 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382cb2f6-5bd3-4acc-8f50-e8f531812280 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.515362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cb2f6-5bd3-4acc-8f50-e8f531812280 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:02.515655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cb2f6-5bd3-4acc-8f50-e8f531812280 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.517238 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382cb2f6-5bd3-4acc-8f50-e8f531812280 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.529031 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3f005-e5c5-4394-9599-61bd3c849346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b4974b4192c453e8824af0c0b2aaaab, user_id=b9b0860b541a43278543b82b9e775bbb, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.529322 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 787/1570] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:12:02.531790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b3f005-e5c5-4394-9599-61bd3c849346 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.534084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cb2f6-5bd3-4acc-8f50-e8f531812280 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.545241 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382cb2f6-5bd3-4acc-8f50-e8f531812280 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.546311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 784/1571] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:02 2022] PUT /identity/v3/projects/1b4974b4192c453e8824af0c0b2aaaab/users/b9b0860b541a43278543b82b9e775bbb/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:02.546914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3f005-e5c5-4394-9599-61bd3c849346 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.603040 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d5f97-5634-43e6-895d-234e49ded0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.603795 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d5f97-5634-43e6-895d-234e49ded0d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.603962 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d5f97-5634-43e6-895d-234e49ded0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.604743 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5d5f97-5634-43e6-895d-234e49ded0d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.605441 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d5f97-5634-43e6-895d-234e49ded0d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.607334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5d5f97-5634-43e6-895d-234e49ded0d7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.615216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b3f005-e5c5-4394-9599-61bd3c849346 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.619831 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294a1e3-df9c-4de2-8910-c77fb4ef4dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.621083 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294a1e3-df9c-4de2-8910-c77fb4ef4dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.621387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294a1e3-df9c-4de2-8910-c77fb4ef4dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.621656 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e294a1e3-df9c-4de2-8910-c77fb4ef4dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.622278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294a1e3-df9c-4de2-8910-c77fb4ef4dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.623904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e294a1e3-df9c-4de2-8910-c77fb4ef4dd4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.634337 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 785/1572] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:02.635113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294a1e3-df9c-4de2-8910-c77fb4ef4dd4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.635520 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 788/1573] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/projects => generated 389 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:02.636407 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d5f97-5634-43e6-895d-234e49ded0d7 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.645527 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e294a1e3-df9c-4de2-8910-c77fb4ef4dd4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.699432 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5d5f97-5634-43e6-895d-234e49ded0d7 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.703127 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c35a7b-27f4-4800-843e-42e30047a156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.703739 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c35a7b-27f4-4800-843e-42e30047a156 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.704017 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c35a7b-27f4-4800-843e-42e30047a156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.704288 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c35a7b-27f4-4800-843e-42e30047a156 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.704445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff642db5-f2a6-473a-aea4-a77f6e2a2400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.705027 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c35a7b-27f4-4800-843e-42e30047a156 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.705128 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff642db5-f2a6-473a-aea4-a77f6e2a2400 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.705349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff642db5-f2a6-473a-aea4-a77f6e2a2400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.705619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff642db5-f2a6-473a-aea4-a77f6e2a2400 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.706241 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff642db5-f2a6-473a-aea4-a77f6e2a2400 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.706966 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c35a7b-27f4-4800-843e-42e30047a156 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.708122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff642db5-f2a6-473a-aea4-a77f6e2a2400 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.738915 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 786/1574] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:02.738915 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c35a7b-27f4-4800-843e-42e30047a156 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.739243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 789/1575] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/projects => generated 377 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:02.739243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff642db5-f2a6-473a-aea4-a77f6e2a2400 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.750485 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:02.753659 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:02.802527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff642db5-f2a6-473a-aea4-a77f6e2a2400 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.805161 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c35a7b-27f4-4800-843e-42e30047a156 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.807111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213599f2-bc40-4056-8ca5-0425d7215b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.807835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213599f2-bc40-4056-8ca5-0425d7215b72 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.808111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213599f2-bc40-4056-8ca5-0425d7215b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.808391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213599f2-bc40-4056-8ca5-0425d7215b72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.809010 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213599f2-bc40-4056-8ca5-0425d7215b72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.810012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d50575-e7b2-42b1-bc9c-6e9005779736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.810611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d50575-e7b2-42b1-bc9c-6e9005779736 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.810890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d50575-e7b2-42b1-bc9c-6e9005779736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.810993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213599f2-bc40-4056-8ca5-0425d7215b72 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.811164 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d50575-e7b2-42b1-bc9c-6e9005779736 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.811778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d50575-e7b2-42b1-bc9c-6e9005779736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.813691 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d50575-e7b2-42b1-bc9c-6e9005779736 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:02.839607 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 790/1576] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/projects => generated 363 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:02.841985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 787/1577] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:02.842493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213599f2-bc40-4056-8ca5-0425d7215b72 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.843616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2cda1-0627-4191-bb83-c160ac2e3b6b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.844020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2cda1-0627-4191-bb83-c160ac2e3b6b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.844342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d2cda1-0627-4191-bb83-c160ac2e3b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.875580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-57d2cda1-0627-4191-bb83-c160ac2e3b6b None None] MFA Rules not processed for user `b9b0860b541a43278543b82b9e775bbb`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:02.876803 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-57d2cda1-0627-4191-bb83-c160ac2e3b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:02.914620 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213599f2-bc40-4056-8ca5-0425d7215b72 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:02.943796 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 788/1578] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:02.946052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2cda1-0627-4191-bb83-c160ac2e3b6b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:02.971289 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d6e76-6a47-4d47-ae93-73bb2c659a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:02.972093 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d6e76-6a47-4d47-ae93-73bb2c659a53 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:02.972968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d6e76-6a47-4d47-ae93-73bb2c659a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:02.972968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726d6e76-6a47-4d47-ae93-73bb2c659a53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:02.973719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d6e76-6a47-4d47-ae93-73bb2c659a53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:02.975675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726d6e76-6a47-4d47-ae93-73bb2c659a53 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.007247 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 791/1579] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/projects => generated 363 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:03.008170 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d6e76-6a47-4d47-ae93-73bb2c659a53 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.016175 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d2cda1-0627-4191-bb83-c160ac2e3b6b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.040070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:03.050221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:03.073617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726d6e76-6a47-4d47-ae93-73bb2c659a53 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.077109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139dd7c-37ca-45e5-8406-1c761807575e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.077744 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139dd7c-37ca-45e5-8406-1c761807575e admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.078023 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139dd7c-37ca-45e5-8406-1c761807575e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.078176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6136df8-ce4f-4998-80c5-95d993622fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.078302 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139dd7c-37ca-45e5-8406-1c761807575e admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.078820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6136df8-ce4f-4998-80c5-95d993622fec admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.078940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7139dd7c-37ca-45e5-8406-1c761807575e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.079089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6136df8-ce4f-4998-80c5-95d993622fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.079358 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6136df8-ce4f-4998-80c5-95d993622fec admin admin] PATH_INFO: `/v3/projects/fee4abdf79dd40c5b7df53a0dbb6cc0f/users/919e0a47fcb745229a7d3c7d1f10dc8e/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.080786 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7139dd7c-37ca-45e5-8406-1c761807575e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.095564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6136df8-ce4f-4998-80c5-95d993622fec admin admin] RBAC: Authorizing `identity:create_grant(project_id=fee4abdf79dd40c5b7df53a0dbb6cc0f, user_id=919e0a47fcb745229a7d3c7d1f10dc8e, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.096810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: WARNING py.warnings [None req-a6136df8-ce4f-4998-80c5-95d993622fec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:12:03.096810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: warnings.warn(msg) Aug 31 19:12:03.096810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: Aug 31 19:12:03.099154 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6136df8-ce4f-4998-80c5-95d993622fec admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.105018 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 789/1580] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:02 2022] POST /identity/v3/users => generated 295 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:03.108291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139dd7c-37ca-45e5-8406-1c761807575e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.113392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 792/1581] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:03 2022] PUT /identity/v3/projects/fee4abdf79dd40c5b7df53a0dbb6cc0f/users/919e0a47fcb745229a7d3c7d1f10dc8e/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:03.124052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6136df8-ce4f-4998-80c5-95d993622fec admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.124203 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139dd7c-37ca-45e5-8406-1c761807575e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.173868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cae6c5-43c4-4124-b7ce-95563a4dd12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.174654 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cae6c5-43c4-4124-b7ce-95563a4dd12c admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.175031 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cae6c5-43c4-4124-b7ce-95563a4dd12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.175427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cae6c5-43c4-4124-b7ce-95563a4dd12c admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.176154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cae6c5-43c4-4124-b7ce-95563a4dd12c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.176352 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6136df8-ce4f-4998-80c5-95d993622fec admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.178291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cae6c5-43c4-4124-b7ce-95563a4dd12c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.181009 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845d4c1-7a65-47ab-9f37-33b18f71a128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.181637 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845d4c1-7a65-47ab-9f37-33b18f71a128 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.181915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845d4c1-7a65-47ab-9f37-33b18f71a128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.182184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845d4c1-7a65-47ab-9f37-33b18f71a128 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.182905 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845d4c1-7a65-47ab-9f37-33b18f71a128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.184784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845d4c1-7a65-47ab-9f37-33b18f71a128 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.205476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 790/1582] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:03 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:03.206445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cae6c5-43c4-4124-b7ce-95563a4dd12c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.206612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 793/1583] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:03 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:03.207402 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845d4c1-7a65-47ab-9f37-33b18f71a128 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.218718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845d4c1-7a65-47ab-9f37-33b18f71a128 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.219466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cae6c5-43c4-4124-b7ce-95563a4dd12c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.223365 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471180e5-e377-43c9-a67e-e5ab5aa25306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.223900 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471180e5-e377-43c9-a67e-e5ab5aa25306 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.224173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471180e5-e377-43c9-a67e-e5ab5aa25306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.224269 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa911ae1-764c-426c-85b7-4ede97a3bd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.224441 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471180e5-e377-43c9-a67e-e5ab5aa25306 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.224774 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa911ae1-764c-426c-85b7-4ede97a3bd36 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.225054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa911ae1-764c-426c-85b7-4ede97a3bd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.225155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471180e5-e377-43c9-a67e-e5ab5aa25306 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.225365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa911ae1-764c-426c-85b7-4ede97a3bd36 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.225986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa911ae1-764c-426c-85b7-4ede97a3bd36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.227001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471180e5-e377-43c9-a67e-e5ab5aa25306 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.227942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa911ae1-764c-426c-85b7-4ede97a3bd36 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.249576 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 794/1584] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:03 2022] POST /identity/v3/users => generated 292 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:03.250788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac7684-35f5-4a17-aa38-97ac22537e25 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.251065 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac7684-35f5-4a17-aa38-97ac22537e25 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.251328 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac7684-35f5-4a17-aa38-97ac22537e25 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.251728 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 791/1585] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:03 2022] POST /identity/v3/users => generated 282 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:03.252405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa911ae1-764c-426c-85b7-4ede97a3bd36 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.256400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 795/1586] 10.222.0.65 () {54 vars in 962 bytes} [Wed Aug 31 19:12:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:03.258028 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac7684-35f5-4a17-aa38-97ac22537e25 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.262821 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa911ae1-764c-426c-85b7-4ede97a3bd36 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.267348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37add8-4b47-4c9d-bf04-895019855ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.267933 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37add8-4b47-4c9d-bf04-895019855ec4 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.268212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37add8-4b47-4c9d-bf04-895019855ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.268495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37add8-4b47-4c9d-bf04-895019855ec4 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.269149 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37add8-4b47-4c9d-bf04-895019855ec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.270975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac7684-35f5-4a17-aa38-97ac22537e25 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.271120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37add8-4b47-4c9d-bf04-895019855ec4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.275512 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef2024-eb32-4ed6-bb34-16b03359d6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.276077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef2024-eb32-4ed6-bb34-16b03359d6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.277015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef2024-eb32-4ed6-bb34-16b03359d6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.277335 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef2024-eb32-4ed6-bb34-16b03359d6b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.277964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef2024-eb32-4ed6-bb34-16b03359d6b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.279558 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef2024-eb32-4ed6-bb34-16b03359d6b0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.291269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 796/1587] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:03.292470 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e889551-7498-4f54-8816-c8ed62a00c85 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.292756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e889551-7498-4f54-8816-c8ed62a00c85 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.293026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e889551-7498-4f54-8816-c8ed62a00c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.296699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 792/1588] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:03 2022] POST /identity/v3/users => generated 282 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:03.296699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37add8-4b47-4c9d-bf04-895019855ec4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.305657 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37add8-4b47-4c9d-bf04-895019855ec4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.310263 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70149def-a343-42bf-87ea-7d51fda45f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.310836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70149def-a343-42bf-87ea-7d51fda45f44 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.311104 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70149def-a343-42bf-87ea-7d51fda45f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.311368 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70149def-a343-42bf-87ea-7d51fda45f44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.311987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70149def-a343-42bf-87ea-7d51fda45f44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.313582 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70149def-a343-42bf-87ea-7d51fda45f44 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.320681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-7e889551-7498-4f54-8816-c8ed62a00c85 None None] MFA Rules not processed for user `919e0a47fcb745229a7d3c7d1f10dc8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:03.322116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7e889551-7498-4f54-8816-c8ed62a00c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:03.326096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 793/1589] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:03.326987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70149def-a343-42bf-87ea-7d51fda45f44 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.338299 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70149def-a343-42bf-87ea-7d51fda45f44 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.343656 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60ee25-efec-47fe-86ef-a2caab82f53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.344242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60ee25-efec-47fe-86ef-a2caab82f53f admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.344517 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60ee25-efec-47fe-86ef-a2caab82f53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.344782 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c60ee25-efec-47fe-86ef-a2caab82f53f admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.345441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60ee25-efec-47fe-86ef-a2caab82f53f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.347082 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c60ee25-efec-47fe-86ef-a2caab82f53f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.357259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 794/1590] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:03.361275 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60ee25-efec-47fe-86ef-a2caab82f53f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.371125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c60ee25-efec-47fe-86ef-a2caab82f53f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.375958 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d358e9a-cc5d-45ec-86f7-a8a2b020b62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.376525 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d358e9a-cc5d-45ec-86f7-a8a2b020b62d admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.376801 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d358e9a-cc5d-45ec-86f7-a8a2b020b62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.377093 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d358e9a-cc5d-45ec-86f7-a8a2b020b62d admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.377735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d358e9a-cc5d-45ec-86f7-a8a2b020b62d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.379279 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d358e9a-cc5d-45ec-86f7-a8a2b020b62d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.389205 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 795/1591] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:03.389982 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d358e9a-cc5d-45ec-86f7-a8a2b020b62d admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.393301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 797/1592] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:03 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:03.393301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71101a63-69a1-41df-b019-d09f86960e0c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.393301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71101a63-69a1-41df-b019-d09f86960e0c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.393301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71101a63-69a1-41df-b019-d09f86960e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.399923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:03.402873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d358e9a-cc5d-45ec-86f7-a8a2b020b62d admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.407425 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055d75e-7a87-4497-a4b1-27d8c2be7a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.408056 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055d75e-7a87-4497-a4b1-27d8c2be7a41 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.408400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055d75e-7a87-4497-a4b1-27d8c2be7a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.409464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9055d75e-7a87-4497-a4b1-27d8c2be7a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.410446 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055d75e-7a87-4497-a4b1-27d8c2be7a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.412155 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9055d75e-7a87-4497-a4b1-27d8c2be7a41 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.422598 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 796/1593] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:03.426274 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055d75e-7a87-4497-a4b1-27d8c2be7a41 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.433182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-71101a63-69a1-41df-b019-d09f86960e0c None None] MFA Rules not processed for user `673a72ac9a9847e3812007b66ac7a5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:03.434412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-71101a63-69a1-41df-b019-d09f86960e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:03.435949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9055d75e-7a87-4497-a4b1-27d8c2be7a41 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.440708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75fc43-9a28-4a1e-92b0-6d60ac18f998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.441309 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75fc43-9a28-4a1e-92b0-6d60ac18f998 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.441588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75fc43-9a28-4a1e-92b0-6d60ac18f998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.441998 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db75fc43-9a28-4a1e-92b0-6d60ac18f998 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.442675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75fc43-9a28-4a1e-92b0-6d60ac18f998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.444235 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db75fc43-9a28-4a1e-92b0-6d60ac18f998 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.454139 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 797/1594] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:03.455089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75fc43-9a28-4a1e-92b0-6d60ac18f998 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.464969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db75fc43-9a28-4a1e-92b0-6d60ac18f998 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.469843 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e255a62-5be2-403f-9deb-292872680573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.470405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e255a62-5be2-403f-9deb-292872680573 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.470678 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e255a62-5be2-403f-9deb-292872680573 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.470945 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e255a62-5be2-403f-9deb-292872680573 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.471550 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e255a62-5be2-403f-9deb-292872680573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.473099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e255a62-5be2-403f-9deb-292872680573 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.484425 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 798/1595] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:03.485659 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e255a62-5be2-403f-9deb-292872680573 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.494802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e255a62-5be2-403f-9deb-292872680573 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.500382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cb23e-fa7c-4e17-9c8c-1f76e3aa6082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.500944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cb23e-fa7c-4e17-9c8c-1f76e3aa6082 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.501249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cb23e-fa7c-4e17-9c8c-1f76e3aa6082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.501513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69cb23e-fa7c-4e17-9c8c-1f76e3aa6082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.502120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69cb23e-fa7c-4e17-9c8c-1f76e3aa6082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.503660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69cb23e-fa7c-4e17-9c8c-1f76e3aa6082 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.507606 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 798/1596] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Aug 31 19:12:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:03.509051 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71101a63-69a1-41df-b019-d09f86960e0c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.516503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 799/1597] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:03.516503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cb23e-fa7c-4e17-9c8c-1f76e3aa6082 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.518962 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71101a63-69a1-41df-b019-d09f86960e0c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.519324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:03.523657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bab33-8662-4a4c-95c4-f3a914f2e0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.524250 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bab33-8662-4a4c-95c4-f3a914f2e0f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.524521 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bab33-8662-4a4c-95c4-f3a914f2e0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.524786 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bab33-8662-4a4c-95c4-f3a914f2e0f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.524950 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69cb23e-fa7c-4e17-9c8c-1f76e3aa6082 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.525443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9bab33-8662-4a4c-95c4-f3a914f2e0f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.527038 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9bab33-8662-4a4c-95c4-f3a914f2e0f3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.531058 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc16623-3b11-405c-bbc3-2691a4f7fb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.531638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc16623-3b11-405c-bbc3-2691a4f7fb9a admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.531911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc16623-3b11-405c-bbc3-2691a4f7fb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.532183 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc16623-3b11-405c-bbc3-2691a4f7fb9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.532806 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc16623-3b11-405c-bbc3-2691a4f7fb9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.534395 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc16623-3b11-405c-bbc3-2691a4f7fb9a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.539293 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 799/1598] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:03.539293 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bab33-8662-4a4c-95c4-f3a914f2e0f3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.546713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 800/1599] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:03.548856 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9bab33-8662-4a4c-95c4-f3a914f2e0f3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.549865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc16623-3b11-405c-bbc3-2691a4f7fb9a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.553796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dacf56-1e54-4b49-a5e7-d47eb78e3ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.554373 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dacf56-1e54-4b49-a5e7-d47eb78e3ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.554647 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dacf56-1e54-4b49-a5e7-d47eb78e3ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.554912 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dacf56-1e54-4b49-a5e7-d47eb78e3ea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.555515 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dacf56-1e54-4b49-a5e7-d47eb78e3ea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.558035 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dacf56-1e54-4b49-a5e7-d47eb78e3ea5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.560532 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc16623-3b11-405c-bbc3-2691a4f7fb9a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.565205 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ad56e-df51-47fe-8066-5e6d0be5a214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.565798 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ad56e-df51-47fe-8066-5e6d0be5a214 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.566079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ad56e-df51-47fe-8066-5e6d0be5a214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.566353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ad56e-df51-47fe-8066-5e6d0be5a214 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.566975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ad56e-df51-47fe-8066-5e6d0be5a214 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.568543 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ad56e-df51-47fe-8066-5e6d0be5a214 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.579679 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 801/1600] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:03.580588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ad56e-df51-47fe-8066-5e6d0be5a214 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.587676 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 800/1601] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:03 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:03.588498 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dacf56-1e54-4b49-a5e7-d47eb78e3ea5 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.590599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ad56e-df51-47fe-8066-5e6d0be5a214 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.647452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5edbe2-7768-4be4-ab53-f37a33cffe0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.648128 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5edbe2-7768-4be4-ab53-f37a33cffe0f admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.648410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5edbe2-7768-4be4-ab53-f37a33cffe0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.648688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5edbe2-7768-4be4-ab53-f37a33cffe0f admin admin] PATH_INFO: `/v3/projects/e3aa913bb25c45039643ce87bb84cbc2/users/aa81a686e4324370a20e3fc35e5cc7a5/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.651913 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dacf56-1e54-4b49-a5e7-d47eb78e3ea5 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.656953 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd7d19-ec6c-4f79-9fc6-edf0cee8a8aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.657585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd7d19-ec6c-4f79-9fc6-edf0cee8a8aa admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.657854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd7d19-ec6c-4f79-9fc6-edf0cee8a8aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.658116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd7d19-ec6c-4f79-9fc6-edf0cee8a8aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.658737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd7d19-ec6c-4f79-9fc6-edf0cee8a8aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.660311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fd7d19-ec6c-4f79-9fc6-edf0cee8a8aa admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.664423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5edbe2-7768-4be4-ab53-f37a33cffe0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3aa913bb25c45039643ce87bb84cbc2, user_id=aa81a686e4324370a20e3fc35e5cc7a5, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.667229 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5edbe2-7768-4be4-ab53-f37a33cffe0f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.670461 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 801/1602] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:03.674851 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd7d19-ec6c-4f79-9fc6-edf0cee8a8aa admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.679859 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 802/1603] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:03 2022] PUT /identity/v3/projects/e3aa913bb25c45039643ce87bb84cbc2/users/aa81a686e4324370a20e3fc35e5cc7a5/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:03.684140 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5edbe2-7768-4be4-ab53-f37a33cffe0f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.688258 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd7d19-ec6c-4f79-9fc6-edf0cee8a8aa admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.730176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:03.746533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86628098-4c18-4292-813e-fda5c3359d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.747098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86628098-4c18-4292-813e-fda5c3359d08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.747370 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86628098-4c18-4292-813e-fda5c3359d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.747636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86628098-4c18-4292-813e-fda5c3359d08 admin admin] PATH_INFO: `/v3/projects/ba8782e8b0af4a2eb7335a0500716b52/users/b4869c70eac84fddb30090585bc1f0de/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.757261 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5edbe2-7768-4be4-ab53-f37a33cffe0f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.762421 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86628098-4c18-4292-813e-fda5c3359d08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba8782e8b0af4a2eb7335a0500716b52, user_id=b4869c70eac84fddb30090585bc1f0de, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.763641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e1377-3676-455f-99e0-3b98f11efe3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.764342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e1377-3676-455f-99e0-3b98f11efe3d service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.764718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e1377-3676-455f-99e0-3b98f11efe3d service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.765051 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86628098-4c18-4292-813e-fda5c3359d08 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.765160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e1377-3676-455f-99e0-3b98f11efe3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.778077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 802/1604] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:03 2022] PUT /identity/v3/projects/ba8782e8b0af4a2eb7335a0500716b52/users/b4869c70eac84fddb30090585bc1f0de/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:03.779069 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86628098-4c18-4292-813e-fda5c3359d08 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.822681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e1377-3676-455f-99e0-3b98f11efe3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.825278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e1377-3676-455f-99e0-3b98f11efe3d service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.829783 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:03.835191 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 803/1605] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:03 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:03.835886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e1377-3676-455f-99e0-3b98f11efe3d service nova] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.841123 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86628098-4c18-4292-813e-fda5c3359d08 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.846594 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cc405-a33b-46aa-a4b2-54488a8a0edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.846973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e1377-3676-455f-99e0-3b98f11efe3d service nova] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.847255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cc405-a33b-46aa-a4b2-54488a8a0edc admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.847532 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cc405-a33b-46aa-a4b2-54488a8a0edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.848110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cc405-a33b-46aa-a4b2-54488a8a0edc admin admin] PATH_INFO: `/v3/projects/a4dece16c7044a0883d30c1234adda44/users/7dfd0bfbf2c0457c96725f0e737b0885/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.851828 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311543c-0586-4a99-90e6-cd003b24fcd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.852454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311543c-0586-4a99-90e6-cd003b24fcd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.852750 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311543c-0586-4a99-90e6-cd003b24fcd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.852994 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4311543c-0586-4a99-90e6-cd003b24fcd1 admin admin] PATH_INFO: `/v3/projects/e6d782400f1e432c8d10a31334c42e68/users/0bfa1533b0944bd681f4c2a33cf701b0/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.867456 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02cc405-a33b-46aa-a4b2-54488a8a0edc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4dece16c7044a0883d30c1234adda44, user_id=7dfd0bfbf2c0457c96725f0e737b0885, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.867851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311543c-0586-4a99-90e6-cd003b24fcd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6d782400f1e432c8d10a31334c42e68, user_id=0bfa1533b0944bd681f4c2a33cf701b0, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.871023 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02cc405-a33b-46aa-a4b2-54488a8a0edc admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.871157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4311543c-0586-4a99-90e6-cd003b24fcd1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.884889 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 804/1606] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:03 2022] PUT /identity/v3/projects/e6d782400f1e432c8d10a31334c42e68/users/0bfa1533b0944bd681f4c2a33cf701b0/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:03.885507 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 803/1607] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:03 2022] PUT /identity/v3/projects/a4dece16c7044a0883d30c1234adda44/users/7dfd0bfbf2c0457c96725f0e737b0885/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:03.893825 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311543c-0586-4a99-90e6-cd003b24fcd1 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.893945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cc405-a33b-46aa-a4b2-54488a8a0edc admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:03.962626 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4311543c-0586-4a99-90e6-cd003b24fcd1 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.967698 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23be40f-c46c-4108-9ee6-443c1e452816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.968291 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cc405-a33b-46aa-a4b2-54488a8a0edc admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:03.968542 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23be40f-c46c-4108-9ee6-443c1e452816 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.968823 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23be40f-c46c-4108-9ee6-443c1e452816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.969249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23be40f-c46c-4108-9ee6-443c1e452816 admin admin] PATH_INFO: `/v3/projects/4c81000f401e4acfbb5264e1834f8fb7/users/5ff8897a5def4518a310ce8441eb4efb/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.973159 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bbb4e4-48c1-46f3-9335-84bfb27b25b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:03.973852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbb4e4-48c1-46f3-9335-84bfb27b25b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:03.974225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbb4e4-48c1-46f3-9335-84bfb27b25b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:03.974578 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bbb4e4-48c1-46f3-9335-84bfb27b25b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:03.975266 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bbb4e4-48c1-46f3-9335-84bfb27b25b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.977265 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bbb4e4-48c1-46f3-9335-84bfb27b25b5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:03.984710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23be40f-c46c-4108-9ee6-443c1e452816 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c81000f401e4acfbb5264e1834f8fb7, user_id=5ff8897a5def4518a310ce8441eb4efb, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:03.987652 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23be40f-c46c-4108-9ee6-443c1e452816 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:04.003314 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 804/1608] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:03 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:04.003945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bbb4e4-48c1-46f3-9335-84bfb27b25b5 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:04.010763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 805/1609] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:03 2022] PUT /identity/v3/projects/4c81000f401e4acfbb5264e1834f8fb7/users/5ff8897a5def4518a310ce8441eb4efb/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:04.014480 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d447f-32d0-4d12-aba0-9d3583e47e2e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.014763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d447f-32d0-4d12-aba0-9d3583e47e2e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.015031 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9d447f-32d0-4d12-aba0-9d3583e47e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.021225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bbb4e4-48c1-46f3-9335-84bfb27b25b5 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:04.035722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ec9d447f-32d0-4d12-aba0-9d3583e47e2e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:04.037009 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ec9d447f-32d0-4d12-aba0-9d3583e47e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:04.068914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:04.091589 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6aceda-5c98-446a-8fdb-b8d0a17f6614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:04.092244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aceda-5c98-446a-8fdb-b8d0a17f6614 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.092517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aceda-5c98-446a-8fdb-b8d0a17f6614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.092780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aceda-5c98-446a-8fdb-b8d0a17f6614 admin admin] PATH_INFO: `/v3/projects/ef754536a67849f59a2f5baa3d51411c/users/9b25c27fd33d4e828b234da80a1449c2/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.103629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 806/1610] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:04.105131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2574a-9657-4a98-bb7b-06a8ff31618d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.105445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2574a-9657-4a98-bb7b-06a8ff31618d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.105786 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2574a-9657-4a98-bb7b-06a8ff31618d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.111921 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6aceda-5c98-446a-8fdb-b8d0a17f6614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef754536a67849f59a2f5baa3d51411c, user_id=9b25c27fd33d4e828b234da80a1449c2, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:04.114649 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6aceda-5c98-446a-8fdb-b8d0a17f6614 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:04.127008 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-63d2574a-9657-4a98-bb7b-06a8ff31618d None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:04.128213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-63d2574a-9657-4a98-bb7b-06a8ff31618d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:04.128455 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 805/1611] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:04 2022] PUT /identity/v3/projects/ef754536a67849f59a2f5baa3d51411c/users/9b25c27fd33d4e828b234da80a1449c2/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:04.129779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f97ec-f22e-495e-9f14-36cdeb478604 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.130084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f97ec-f22e-495e-9f14-36cdeb478604 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.130327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f97ec-f22e-495e-9f14-36cdeb478604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.151883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-7a8f97ec-f22e-495e-9f14-36cdeb478604 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:04.153094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7a8f97ec-f22e-495e-9f14-36cdeb478604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:04.185483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:04.197520 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 807/1612] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:04.202061 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779c589-3858-42b3-9263-0f44db1ecb61 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.202350 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779c589-3858-42b3-9263-0f44db1ecb61 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.202770 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779c589-3858-42b3-9263-0f44db1ecb61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.217384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 806/1613] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:04.218382 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f97ec-f22e-495e-9f14-36cdeb478604 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:04.231076 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f97ec-f22e-495e-9f14-36cdeb478604 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:04.232790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8779c589-3858-42b3-9263-0f44db1ecb61 None None] MFA Rules not processed for user `7dfd0bfbf2c0457c96725f0e737b0885`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:04.234003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8779c589-3858-42b3-9263-0f44db1ecb61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:04.236074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1755c91-526f-40ba-8266-d89cb6a679cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:04.236691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1755c91-526f-40ba-8266-d89cb6a679cf admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.236951 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1755c91-526f-40ba-8266-d89cb6a679cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.237272 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1755c91-526f-40ba-8266-d89cb6a679cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.237910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1755c91-526f-40ba-8266-d89cb6a679cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:04.239492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1755c91-526f-40ba-8266-d89cb6a679cf admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:04.250844 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 807/1614] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:04.252272 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e9623-afa3-447c-8de1-5a32c4013882 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.252712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e9623-afa3-447c-8de1-5a32c4013882 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.252980 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55e9623-afa3-447c-8de1-5a32c4013882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.284741 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c55e9623-afa3-447c-8de1-5a32c4013882 None None] MFA Rules not processed for user `5ff8897a5def4518a310ce8441eb4efb`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:04.286558 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c55e9623-afa3-447c-8de1-5a32c4013882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:04.310864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 808/1615] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:04.312202 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53df2e-8f87-4d7b-b4a8-d74cc6ef7355 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.312494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53df2e-8f87-4d7b-b4a8-d74cc6ef7355 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.312770 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed53df2e-8f87-4d7b-b4a8-d74cc6ef7355 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.314409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 809/1616] 10.222.0.65 () {54 vars in 912 bytes} [Wed Aug 31 19:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:04.315175 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53df2e-8f87-4d7b-b4a8-d74cc6ef7355 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:04.355393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 808/1617] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:04.356656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4324b22-1e3a-4847-926a-5eaa4a09520c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.357393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4324b22-1e3a-4847-926a-5eaa4a09520c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.357669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4324b22-1e3a-4847-926a-5eaa4a09520c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.374357 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed53df2e-8f87-4d7b-b4a8-d74cc6ef7355 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:04.378193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bad21d-7551-4a55-af7f-ea008d35d694 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:04.378796 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bad21d-7551-4a55-af7f-ea008d35d694 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.379073 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bad21d-7551-4a55-af7f-ea008d35d694 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.379341 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bad21d-7551-4a55-af7f-ea008d35d694 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.386733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c4324b22-1e3a-4847-926a-5eaa4a09520c None None] MFA Rules not processed for user `9b25c27fd33d4e828b234da80a1449c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:04.388072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c4324b22-1e3a-4847-926a-5eaa4a09520c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:04.388662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bad21d-7551-4a55-af7f-ea008d35d694 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:04.390114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bad21d-7551-4a55-af7f-ea008d35d694 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:04.399828 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 810/1618] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:04.402627 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bad21d-7551-4a55-af7f-ea008d35d694 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:04.411203 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bad21d-7551-4a55-af7f-ea008d35d694 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:04.414988 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41117b39-19c2-472c-bfb0-5f4e97d99f1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:04.415606 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41117b39-19c2-472c-bfb0-5f4e97d99f1f service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.415875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41117b39-19c2-472c-bfb0-5f4e97d99f1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.416148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41117b39-19c2-472c-bfb0-5f4e97d99f1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.425992 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:04.427069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41117b39-19c2-472c-bfb0-5f4e97d99f1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:04.428797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41117b39-19c2-472c-bfb0-5f4e97d99f1f service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:04.443132 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 811/1619] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:04.444371 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41117b39-19c2-472c-bfb0-5f4e97d99f1f service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:04.454193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41117b39-19c2-472c-bfb0-5f4e97d99f1f service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:04.458043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414fb33-1c02-40c4-8dbc-f1fff9767f57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:04.458638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414fb33-1c02-40c4-8dbc-f1fff9767f57 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.458944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414fb33-1c02-40c4-8dbc-f1fff9767f57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.459191 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414fb33-1c02-40c4-8dbc-f1fff9767f57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.469263 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 809/1620] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:04.469263 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4324b22-1e3a-4847-926a-5eaa4a09520c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:04.470237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414fb33-1c02-40c4-8dbc-f1fff9767f57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:04.471894 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7414fb33-1c02-40c4-8dbc-f1fff9767f57 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:04.482466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4324b22-1e3a-4847-926a-5eaa4a09520c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:04.485409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 812/1621] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:04.486256 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414fb33-1c02-40c4-8dbc-f1fff9767f57 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:04.488594 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad34c5d2-f17d-4e1a-9632-6b04c8f8924a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:04.489379 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34c5d2-f17d-4e1a-9632-6b04c8f8924a admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.489745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34c5d2-f17d-4e1a-9632-6b04c8f8924a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.490107 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad34c5d2-f17d-4e1a-9632-6b04c8f8924a admin admin] PATH_INFO: `/v3/projects/bab1237adca246ef9b82148b93c30e9b/users/fe93b54595b84cb08687844f9fb38ec8/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.497810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414fb33-1c02-40c4-8dbc-f1fff9767f57 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:04.503178 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078410f-7e31-4da3-8682-3fe837e2779c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:04.503938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078410f-7e31-4da3-8682-3fe837e2779c admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.504228 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078410f-7e31-4da3-8682-3fe837e2779c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.504727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6078410f-7e31-4da3-8682-3fe837e2779c admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.506437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078410f-7e31-4da3-8682-3fe837e2779c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:04.508897 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6078410f-7e31-4da3-8682-3fe837e2779c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:04.509696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34c5d2-f17d-4e1a-9632-6b04c8f8924a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bab1237adca246ef9b82148b93c30e9b, user_id=fe93b54595b84cb08687844f9fb38ec8, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:04.513092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad34c5d2-f17d-4e1a-9632-6b04c8f8924a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:04.530241 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 810/1622] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:04 2022] PUT /identity/v3/projects/bab1237adca246ef9b82148b93c30e9b/users/fe93b54595b84cb08687844f9fb38ec8/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:04.531606 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744a952-db4d-418e-8a9b-dd7bf8d3d7bc None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.531872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744a952-db4d-418e-8a9b-dd7bf8d3d7bc None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.532391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6744a952-db4d-418e-8a9b-dd7bf8d3d7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.553628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 813/1623] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:04 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:04.553628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078410f-7e31-4da3-8682-3fe837e2779c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:04.555204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6744a952-db4d-418e-8a9b-dd7bf8d3d7bc None None] MFA Rules not processed for user `673a72ac9a9847e3812007b66ac7a5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:04.556453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6744a952-db4d-418e-8a9b-dd7bf8d3d7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:04.614080 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:04.633548 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6078410f-7e31-4da3-8682-3fe837e2779c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:04.640293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47591b35-f746-46a2-b1d4-bc4683a13e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:04.640991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47591b35-f746-46a2-b1d4-bc4683a13e10 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.641304 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47591b35-f746-46a2-b1d4-bc4683a13e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.641581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47591b35-f746-46a2-b1d4-bc4683a13e10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.642367 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47591b35-f746-46a2-b1d4-bc4683a13e10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:04.643404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 811/1624] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 19:12:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:04.644355 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744a952-db4d-418e-8a9b-dd7bf8d3d7bc None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:04.644973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47591b35-f746-46a2-b1d4-bc4683a13e10 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:04.656630 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6744a952-db4d-418e-8a9b-dd7bf8d3d7bc None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:04.662027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb518434-1564-4f88-98a8-da8a20e85b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:04.662637 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb518434-1564-4f88-98a8-da8a20e85b9e admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.662973 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb518434-1564-4f88-98a8-da8a20e85b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.663232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb518434-1564-4f88-98a8-da8a20e85b9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.664181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb518434-1564-4f88-98a8-da8a20e85b9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:04.666534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb518434-1564-4f88-98a8-da8a20e85b9e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:04.679845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 814/1625] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:04 2022] POST /identity/v3/projects => generated 365 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:04.680748 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47591b35-f746-46a2-b1d4-bc4683a13e10 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:04.698974 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 812/1626] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:04 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:04.699979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb518434-1564-4f88-98a8-da8a20e85b9e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:04.756992 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47591b35-f746-46a2-b1d4-bc4683a13e10 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:04.781469 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb518434-1564-4f88-98a8-da8a20e85b9e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:04.787195 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee44caf-8ab3-4a12-b6ce-5c29f9c4e719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:04.788543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee44caf-8ab3-4a12-b6ce-5c29f9c4e719 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.788892 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee44caf-8ab3-4a12-b6ce-5c29f9c4e719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.789258 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee44caf-8ab3-4a12-b6ce-5c29f9c4e719 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.790003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee44caf-8ab3-4a12-b6ce-5c29f9c4e719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:04.792547 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee44caf-8ab3-4a12-b6ce-5c29f9c4e719 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:04.837245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 813/1627] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:04 2022] POST /identity/v3/users => generated 293 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:04.837245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee44caf-8ab3-4a12-b6ce-5c29f9c4e719 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:04.842727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca5c29-4954-4e48-b648-843f34f99e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:04.843261 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca5c29-4954-4e48-b648-843f34f99e9c admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.843508 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca5c29-4954-4e48-b648-843f34f99e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.844384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ca5c29-4954-4e48-b648-843f34f99e9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.844830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca5c29-4954-4e48-b648-843f34f99e9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:04.846461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ca5c29-4954-4e48-b648-843f34f99e9c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:04.848487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:04.858244 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 815/1628] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:04.859021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca5c29-4954-4e48-b648-843f34f99e9c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:04.871459 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ca5c29-4954-4e48-b648-843f34f99e9c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:04.877039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b6270-c1a9-47f7-a7c8-16f9eec66a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:04.877675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b6270-c1a9-47f7-a7c8-16f9eec66a14 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.878202 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b6270-c1a9-47f7-a7c8-16f9eec66a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.878202 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b6270-c1a9-47f7-a7c8-16f9eec66a14 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.878817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b6270-c1a9-47f7-a7c8-16f9eec66a14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:04.881069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b6270-c1a9-47f7-a7c8-16f9eec66a14 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:04.915740 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee44caf-8ab3-4a12-b6ce-5c29f9c4e719 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:04.921789 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55625dd4-f5ad-47a8-8ee7-300e9fac723f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:04.922825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55625dd4-f5ad-47a8-8ee7-300e9fac723f service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.922825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55625dd4-f5ad-47a8-8ee7-300e9fac723f service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.923098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55625dd4-f5ad-47a8-8ee7-300e9fac723f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.927522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 816/1629] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:04 2022] POST /identity/v3/users => generated 289 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:04.929603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b6270-c1a9-47f7-a7c8-16f9eec66a14 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:04.944845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b6270-c1a9-47f7-a7c8-16f9eec66a14 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:04.954763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c96edb-0ba8-461a-8957-c0c8fe8ae57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:04.955976 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c96edb-0ba8-461a-8957-c0c8fe8ae57e admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:04.956429 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c96edb-0ba8-461a-8957-c0c8fe8ae57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:04.957828 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c96edb-0ba8-461a-8957-c0c8fe8ae57e admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:04.960033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c96edb-0ba8-461a-8957-c0c8fe8ae57e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:04.962563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c96edb-0ba8-461a-8957-c0c8fe8ae57e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:04.999870 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 817/1630] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:04 2022] POST /identity/v3/users => generated 288 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:05.001087 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c96edb-0ba8-461a-8957-c0c8fe8ae57e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:05.017094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c96edb-0ba8-461a-8957-c0c8fe8ae57e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:05.025814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0af544-6129-40c6-9466-7806a61be0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:05.027463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0af544-6129-40c6-9466-7806a61be0f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:05.027799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0af544-6129-40c6-9466-7806a61be0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:05.028194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0af544-6129-40c6-9466-7806a61be0f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:05.029060 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0af544-6129-40c6-9466-7806a61be0f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:05.031709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0af544-6129-40c6-9466-7806a61be0f9 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:05.046389 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 818/1631] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:05.047503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0af544-6129-40c6-9466-7806a61be0f9 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:05.060453 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0af544-6129-40c6-9466-7806a61be0f9 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:05.063499 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55625dd4-f5ad-47a8-8ee7-300e9fac723f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:05.066200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d9b06-9605-4572-a55d-bd42aaee2583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:05.066604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55625dd4-f5ad-47a8-8ee7-300e9fac723f service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:05.067009 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d9b06-9605-4572-a55d-bd42aaee2583 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:05.067291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d9b06-9605-4572-a55d-bd42aaee2583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:05.067582 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490d9b06-9605-4572-a55d-bd42aaee2583 admin admin] PATH_INFO: `/v3/projects/bab1237adca246ef9b82148b93c30e9b/users/fe93b54595b84cb08687844f9fb38ec8/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:05.069681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d9b06-9605-4572-a55d-bd42aaee2583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bab1237adca246ef9b82148b93c30e9b, user_id=fe93b54595b84cb08687844f9fb38ec8, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:05.072810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490d9b06-9605-4572-a55d-bd42aaee2583 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:05.083805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 814/1632] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:05.084857 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55625dd4-f5ad-47a8-8ee7-300e9fac723f service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:05.096732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 819/1633] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:05 2022] PUT /identity/v3/projects/bab1237adca246ef9b82148b93c30e9b/users/fe93b54595b84cb08687844f9fb38ec8/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:05.096732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d9b06-9605-4572-a55d-bd42aaee2583 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:05.097057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55625dd4-f5ad-47a8-8ee7-300e9fac723f service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:05.108730 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:05.173277 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490d9b06-9605-4572-a55d-bd42aaee2583 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:05.177906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fadc02-3ae9-48e0-91a1-2784e5375967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:05.178527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fadc02-3ae9-48e0-91a1-2784e5375967 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:05.178802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fadc02-3ae9-48e0-91a1-2784e5375967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:05.179044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed76668-945b-455f-b1a7-f6ec2b555013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:05.179214 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fadc02-3ae9-48e0-91a1-2784e5375967 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:05.179659 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed76668-945b-455f-b1a7-f6ec2b555013 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:05.179775 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fadc02-3ae9-48e0-91a1-2784e5375967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:05.179939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed76668-945b-455f-b1a7-f6ec2b555013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:05.180218 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed76668-945b-455f-b1a7-f6ec2b555013 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:05.180855 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed76668-945b-455f-b1a7-f6ec2b555013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:05.181490 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fadc02-3ae9-48e0-91a1-2784e5375967 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:05.182736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed76668-945b-455f-b1a7-f6ec2b555013 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:05.196934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 820/1634] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:05.198509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed76668-945b-455f-b1a7-f6ec2b555013 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:05.200853 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 815/1635] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:05.203368 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089801a-6260-4e3b-9611-40e1ca52210e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:05.203645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089801a-6260-4e3b-9611-40e1ca52210e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:05.203916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3089801a-6260-4e3b-9611-40e1ca52210e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:05.213454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed76668-945b-455f-b1a7-f6ec2b555013 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:05.216196 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9beff84-694f-44b5-87bf-500c1fd10952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:05.216340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9beff84-694f-44b5-87bf-500c1fd10952 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:05.216625 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9beff84-694f-44b5-87bf-500c1fd10952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:05.216981 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9beff84-694f-44b5-87bf-500c1fd10952 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:05.217849 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9beff84-694f-44b5-87bf-500c1fd10952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:05.219745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9beff84-694f-44b5-87bf-500c1fd10952 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:05.231673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 821/1636] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:05.232835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9beff84-694f-44b5-87bf-500c1fd10952 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:05.243693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3089801a-6260-4e3b-9611-40e1ca52210e None None] MFA Rules not processed for user `fe93b54595b84cb08687844f9fb38ec8`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:05.244811 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9beff84-694f-44b5-87bf-500c1fd10952 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:05.245369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3089801a-6260-4e3b-9611-40e1ca52210e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:05.250153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea7944-c186-4163-9d2f-efe33f103c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:05.250835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea7944-c186-4163-9d2f-efe33f103c3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:05.251160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea7944-c186-4163-9d2f-efe33f103c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:05.251440 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea7944-c186-4163-9d2f-efe33f103c3e admin admin] PATH_INFO: `/v3/projects/f8f4c294af4641a88e71e093731cad59/users/2947bd5cf3654753914dfe9702295cb5/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:05.269011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea7944-c186-4163-9d2f-efe33f103c3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8f4c294af4641a88e71e093731cad59, user_id=2947bd5cf3654753914dfe9702295cb5, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:05.271861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea7944-c186-4163-9d2f-efe33f103c3e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:05.298611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 822/1637] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:05 2022] PUT /identity/v3/projects/f8f4c294af4641a88e71e093731cad59/users/2947bd5cf3654753914dfe9702295cb5/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:05.300114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea7944-c186-4163-9d2f-efe33f103c3e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:05.323757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:05.398519 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 816/1638] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:05.402275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089801a-6260-4e3b-9611-40e1ca52210e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:05.444407 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3089801a-6260-4e3b-9611-40e1ca52210e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:05.456397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea7944-c186-4163-9d2f-efe33f103c3e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:05.459191 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceba29-1138-450e-aeab-82fb58be39c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:05.459878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceba29-1138-450e-aeab-82fb58be39c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:05.460155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceba29-1138-450e-aeab-82fb58be39c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:05.460428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ceba29-1138-450e-aeab-82fb58be39c3 admin admin] PATH_INFO: `/v3/projects/e7a629d72e234343aaf34e8627bf8124/users/b302a95cd1404b1b80d534db57e78b3e/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:05.483911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12889afd-bb98-4dbd-91cc-70d95cbd0f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:05.487577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12889afd-bb98-4dbd-91cc-70d95cbd0f9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:05.489045 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12889afd-bb98-4dbd-91cc-70d95cbd0f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:05.489603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12889afd-bb98-4dbd-91cc-70d95cbd0f9a admin admin] PATH_INFO: `/v3/projects/65b58bb980cc49b0a0adfbad99ae4326/users/7deaaeb48b8c40d28a24cf14db40c07c/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:05.520475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ceba29-1138-450e-aeab-82fb58be39c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7a629d72e234343aaf34e8627bf8124, user_id=b302a95cd1404b1b80d534db57e78b3e, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:05.523160 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ceba29-1138-450e-aeab-82fb58be39c3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:05.523793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12889afd-bb98-4dbd-91cc-70d95cbd0f9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b58bb980cc49b0a0adfbad99ae4326, user_id=7deaaeb48b8c40d28a24cf14db40c07c, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:05.527879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12889afd-bb98-4dbd-91cc-70d95cbd0f9a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:05.574871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 823/1639] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:05 2022] PUT /identity/v3/projects/65b58bb980cc49b0a0adfbad99ae4326/users/7deaaeb48b8c40d28a24cf14db40c07c/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:05.574871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12889afd-bb98-4dbd-91cc-70d95cbd0f9a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:05.577792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 817/1640] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:05 2022] PUT /identity/v3/projects/e7a629d72e234343aaf34e8627bf8124/users/b302a95cd1404b1b80d534db57e78b3e/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:05.586317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceba29-1138-450e-aeab-82fb58be39c3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:05.730669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ceba29-1138-450e-aeab-82fb58be39c3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:05.743765 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c2579-a5a4-470d-a197-9b4b770321b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:05.743765 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c2579-a5a4-470d-a197-9b4b770321b8 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:05.744089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c2579-a5a4-470d-a197-9b4b770321b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:05.744346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390c2579-a5a4-470d-a197-9b4b770321b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:05.772310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:05.793953 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12889afd-bb98-4dbd-91cc-70d95cbd0f9a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:05.899097 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c2579-a5a4-470d-a197-9b4b770321b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:05.900548 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390c2579-a5a4-470d-a197-9b4b770321b8 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:05.922521 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 818/1641] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:05.923482 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c2579-a5a4-470d-a197-9b4b770321b8 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:05.930694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00477136-6b0a-4945-a0d4-85c39bd1dc5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:05.931428 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00477136-6b0a-4945-a0d4-85c39bd1dc5b admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:05.931811 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00477136-6b0a-4945-a0d4-85c39bd1dc5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:05.932189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00477136-6b0a-4945-a0d4-85c39bd1dc5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:05.934105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00477136-6b0a-4945-a0d4-85c39bd1dc5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:05.937475 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00477136-6b0a-4945-a0d4-85c39bd1dc5b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:05.952503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 824/1642] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:05.954554 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b151e2a-d9b1-49ba-a2da-07a0297c6dcd None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:05.954944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b151e2a-d9b1-49ba-a2da-07a0297c6dcd None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:05.955475 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b151e2a-d9b1-49ba-a2da-07a0297c6dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:05.958564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390c2579-a5a4-470d-a197-9b4b770321b8 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:05.975813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d00f70-de1a-46ad-bfcf-845f662c8760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:05.978330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d00f70-de1a-46ad-bfcf-845f662c8760 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:05.978663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d00f70-de1a-46ad-bfcf-845f662c8760 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:05.979742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d00f70-de1a-46ad-bfcf-845f662c8760 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:05.980565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d00f70-de1a-46ad-bfcf-845f662c8760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:05.982213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d00f70-de1a-46ad-bfcf-845f662c8760 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:06.012721 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-6b151e2a-d9b1-49ba-a2da-07a0297c6dcd None None] MFA Rules not processed for user `b302a95cd1404b1b80d534db57e78b3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:06.016684 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6b151e2a-d9b1-49ba-a2da-07a0297c6dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:06.034746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 819/1643] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:06.035615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d00f70-de1a-46ad-bfcf-845f662c8760 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:06.063784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d00f70-de1a-46ad-bfcf-845f662c8760 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:06.079006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ea1f7-0f71-4afe-a3d6-ff890c1df96a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:06.079742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ea1f7-0f71-4afe-a3d6-ff890c1df96a admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:06.080024 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ea1f7-0f71-4afe-a3d6-ff890c1df96a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:06.080299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ea1f7-0f71-4afe-a3d6-ff890c1df96a admin admin] PATH_INFO: `/v3/projects/f8f4c294af4641a88e71e093731cad59/users/2947bd5cf3654753914dfe9702295cb5/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:06.084270 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ea1f7-0f71-4afe-a3d6-ff890c1df96a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8f4c294af4641a88e71e093731cad59, user_id=2947bd5cf3654753914dfe9702295cb5, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:06.086975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ea1f7-0f71-4afe-a3d6-ff890c1df96a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:06.119603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 820/1644] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:06 2022] PUT /identity/v3/projects/f8f4c294af4641a88e71e093731cad59/users/2947bd5cf3654753914dfe9702295cb5/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:06.121066 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ea1f7-0f71-4afe-a3d6-ff890c1df96a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:06.180745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 825/1645] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:06.180745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878fc78-f6a4-43b3-979e-8993dc5ced5e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:06.180745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878fc78-f6a4-43b3-979e-8993dc5ced5e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:06.180745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f878fc78-f6a4-43b3-979e-8993dc5ced5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:06.218833 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f878fc78-f6a4-43b3-979e-8993dc5ced5e None None] MFA Rules not processed for user `2947bd5cf3654753914dfe9702295cb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:06.220548 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f878fc78-f6a4-43b3-979e-8993dc5ced5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:06.255337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:06.314531 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ea1f7-0f71-4afe-a3d6-ff890c1df96a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:06.320303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 826/1646] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:06 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:06.321363 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878fc78-f6a4-43b3-979e-8993dc5ced5e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:06.330583 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d798aeb-a1cb-4b0a-b54c-f0393b28147e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:06.331269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d798aeb-a1cb-4b0a-b54c-f0393b28147e admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:06.331555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d798aeb-a1cb-4b0a-b54c-f0393b28147e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:06.332195 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d798aeb-a1cb-4b0a-b54c-f0393b28147e admin admin] PATH_INFO: `/v3/projects/65b58bb980cc49b0a0adfbad99ae4326/users/7deaaeb48b8c40d28a24cf14db40c07c/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:06.336022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d798aeb-a1cb-4b0a-b54c-f0393b28147e admin admin] RBAC: Authorizing `identity:create_grant(project_id=65b58bb980cc49b0a0adfbad99ae4326, user_id=7deaaeb48b8c40d28a24cf14db40c07c, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:06.343227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:06.345565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d798aeb-a1cb-4b0a-b54c-f0393b28147e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:06.373135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 821/1647] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:06 2022] PUT /identity/v3/projects/65b58bb980cc49b0a0adfbad99ae4326/users/7deaaeb48b8c40d28a24cf14db40c07c/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:06.377523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d798aeb-a1cb-4b0a-b54c-f0393b28147e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:06.399692 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f878fc78-f6a4-43b3-979e-8993dc5ced5e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:06.471165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cbff3-7abe-4cf5-9397-96a3aac47660 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:06.472042 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cbff3-7abe-4cf5-9397-96a3aac47660 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:06.472325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cbff3-7abe-4cf5-9397-96a3aac47660 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:06.472599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78cbff3-7abe-4cf5-9397-96a3aac47660 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:06.515562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d798aeb-a1cb-4b0a-b54c-f0393b28147e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:06.527633 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836dc350-322b-428e-b407-99cf994379ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:06.528256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836dc350-322b-428e-b407-99cf994379ba admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:06.528531 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836dc350-322b-428e-b407-99cf994379ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:06.528799 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836dc350-322b-428e-b407-99cf994379ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:06.529445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836dc350-322b-428e-b407-99cf994379ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:06.531493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836dc350-322b-428e-b407-99cf994379ba admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:06.548167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78cbff3-7abe-4cf5-9397-96a3aac47660 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:06.549662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78cbff3-7abe-4cf5-9397-96a3aac47660 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:06.565460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 827/1648] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:06.566165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cbff3-7abe-4cf5-9397-96a3aac47660 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:06.575222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78cbff3-7abe-4cf5-9397-96a3aac47660 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:06.579482 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ba574-3965-42e2-adff-33262af84fa4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:06.580097 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ba574-3965-42e2-adff-33262af84fa4 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:06.580428 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ba574-3965-42e2-adff-33262af84fa4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:06.580708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ba574-3965-42e2-adff-33262af84fa4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:06.596971 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 822/1649] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:06 2022] POST /identity/v3/projects => generated 387 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:06.596971 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e09223-3e4b-4ddf-96fb-f018da70312d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:06.596971 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e09223-3e4b-4ddf-96fb-f018da70312d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:06.596971 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e09223-3e4b-4ddf-96fb-f018da70312d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:06.651603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ba574-3965-42e2-adff-33262af84fa4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:06.653320 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ba574-3965-42e2-adff-33262af84fa4 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:06.658093 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-94e09223-3e4b-4ddf-96fb-f018da70312d None None] MFA Rules not processed for user `7deaaeb48b8c40d28a24cf14db40c07c`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:06.660586 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-94e09223-3e4b-4ddf-96fb-f018da70312d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:06.674890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 828/1650] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:06 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:06.676677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ba574-3965-42e2-adff-33262af84fa4 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:06.701404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ba574-3965-42e2-adff-33262af84fa4 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:06.745490 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:06.815273 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab593aa2-4cc2-4467-92bb-700816268cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:06.815948 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab593aa2-4cc2-4467-92bb-700816268cf2 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:06.816171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab593aa2-4cc2-4467-92bb-700816268cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:06.816443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab593aa2-4cc2-4467-92bb-700816268cf2 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:06.820466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab593aa2-4cc2-4467-92bb-700816268cf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:06.829248 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab593aa2-4cc2-4467-92bb-700816268cf2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:06.856171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 823/1651] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:06.877092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 829/1652] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:06 2022] POST /identity/v3/users => generated 300 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:06.882617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e09223-3e4b-4ddf-96fb-f018da70312d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:06.893909 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab593aa2-4cc2-4467-92bb-700816268cf2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:06.902909 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e09223-3e4b-4ddf-96fb-f018da70312d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:06.914638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64af5df-e27a-44ab-bb5b-17dfb60807f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:06.916006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64af5df-e27a-44ab-bb5b-17dfb60807f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:06.917169 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64af5df-e27a-44ab-bb5b-17dfb60807f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:06.917730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64af5df-e27a-44ab-bb5b-17dfb60807f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:06.919245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64af5df-e27a-44ab-bb5b-17dfb60807f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:06.922034 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64af5df-e27a-44ab-bb5b-17dfb60807f3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:06.936337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 824/1653] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:06.940298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64af5df-e27a-44ab-bb5b-17dfb60807f3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:06.965662 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64af5df-e27a-44ab-bb5b-17dfb60807f3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:06.983987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf235887-fbe9-4a97-bb3d-241cf39cd8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:06.984827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf235887-fbe9-4a97-bb3d-241cf39cd8dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:06.985472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf235887-fbe9-4a97-bb3d-241cf39cd8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:06.985822 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf235887-fbe9-4a97-bb3d-241cf39cd8dc admin admin] PATH_INFO: `/v3/projects/e46e5e1cbd984396b231d616f6223617/users/35978b8fcccc4b57bb8d4a15032e708a/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:07.006375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf235887-fbe9-4a97-bb3d-241cf39cd8dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e46e5e1cbd984396b231d616f6223617, user_id=35978b8fcccc4b57bb8d4a15032e708a, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:07.009632 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab593aa2-4cc2-4467-92bb-700816268cf2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:07.010113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf235887-fbe9-4a97-bb3d-241cf39cd8dc admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:07.014753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfb27f-2143-4bbb-93f3-bf2cb4921712 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:07.015733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfb27f-2143-4bbb-93f3-bf2cb4921712 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:07.016178 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfb27f-2143-4bbb-93f3-bf2cb4921712 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:07.016631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbfb27f-2143-4bbb-93f3-bf2cb4921712 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:07.021849 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:07.033861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 825/1654] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:06 2022] PUT /identity/v3/projects/e46e5e1cbd984396b231d616f6223617/users/35978b8fcccc4b57bb8d4a15032e708a/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:12:07.033861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7609b-3a5c-458d-bedd-9b3938c9a449 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:07.033861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7609b-3a5c-458d-bedd-9b3938c9a449 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:07.035871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7609b-3a5c-458d-bedd-9b3938c9a449 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:07.036830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbfb27f-2143-4bbb-93f3-bf2cb4921712 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:07.037472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 826/1655] 10.222.0.65 () {54 vars in 962 bytes} [Wed Aug 31 19:12:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:07.038555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7609b-3a5c-458d-bedd-9b3938c9a449 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:07.047177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbfb27f-2143-4bbb-93f3-bf2cb4921712 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:07.065838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 830/1656] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:07.078460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfb27f-2143-4bbb-93f3-bf2cb4921712 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:07.106425 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbfb27f-2143-4bbb-93f3-bf2cb4921712 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:07.133464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c7609b-3a5c-458d-bedd-9b3938c9a449 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:07.139787 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300cac2e-5378-4d8d-9b91-0e44e278b74c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:07.140777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300cac2e-5378-4d8d-9b91-0e44e278b74c service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:07.141261 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300cac2e-5378-4d8d-9b91-0e44e278b74c service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:07.141723 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300cac2e-5378-4d8d-9b91-0e44e278b74c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:07.261540 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300cac2e-5378-4d8d-9b91-0e44e278b74c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:07.263093 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300cac2e-5378-4d8d-9b91-0e44e278b74c service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:07.264195 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533dc33-fcd2-401f-888d-3f9c0f69e58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:07.266511 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533dc33-fcd2-401f-888d-3f9c0f69e58a admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:07.266995 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533dc33-fcd2-401f-888d-3f9c0f69e58a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:07.269734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4533dc33-fcd2-401f-888d-3f9c0f69e58a admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:07.270583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533dc33-fcd2-401f-888d-3f9c0f69e58a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:07.272363 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4533dc33-fcd2-401f-888d-3f9c0f69e58a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:07.274748 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 827/1657] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:07.276397 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22125909-4c96-495f-a1ab-37eede403bd5 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:07.277028 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22125909-4c96-495f-a1ab-37eede403bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:07.277607 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22125909-4c96-495f-a1ab-37eede403bd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:07.304364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-22125909-4c96-495f-a1ab-37eede403bd5 None None] MFA Rules not processed for user `673a72ac9a9847e3812007b66ac7a5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:07.306346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-22125909-4c96-495f-a1ab-37eede403bd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:07.308555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 831/1658] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:07 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:07.312327 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533dc33-fcd2-401f-888d-3f9c0f69e58a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:07.342366 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4533dc33-fcd2-401f-888d-3f9c0f69e58a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:07.355150 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bdb4e-f718-4bbc-9088-4bd5ea2e05f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:07.355824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bdb4e-f718-4bbc-9088-4bd5ea2e05f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:07.356611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bdb4e-f718-4bbc-9088-4bd5ea2e05f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:07.356888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bdb4e-f718-4bbc-9088-4bd5ea2e05f2 admin admin] PATH_INFO: `/v3/projects/e46e5e1cbd984396b231d616f6223617/users/35978b8fcccc4b57bb8d4a15032e708a/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:07.367918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bdb4e-f718-4bbc-9088-4bd5ea2e05f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e46e5e1cbd984396b231d616f6223617, user_id=35978b8fcccc4b57bb8d4a15032e708a, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:07.380279 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0bdb4e-f718-4bbc-9088-4bd5ea2e05f2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:07.413249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 832/1659] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:07 2022] PUT /identity/v3/projects/e46e5e1cbd984396b231d616f6223617/users/35978b8fcccc4b57bb8d4a15032e708a/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:07.414084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 828/1660] 10.222.0.65 () {58 vars in 1059 bytes} [Wed Aug 31 19:12:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:07.429885 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342677f9-23a9-4300-b6e6-43890016723b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:07.430462 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342677f9-23a9-4300-b6e6-43890016723b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:07.430847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342677f9-23a9-4300-b6e6-43890016723b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:07.431066 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1dbab-0fdb-4aa1-9f41-9c7d474ad397 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:07.431448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1dbab-0fdb-4aa1-9f41-9c7d474ad397 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:07.432802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1dbab-0fdb-4aa1-9f41-9c7d474ad397 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:07.442671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 829/1661] 10.222.0.65 () {54 vars in 912 bytes} [Wed Aug 31 19:12:07 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:07.445211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eee792-c138-466e-a884-6e58219975fd None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:07.446574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eee792-c138-466e-a884-6e58219975fd None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:07.446574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eee792-c138-466e-a884-6e58219975fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:07.501475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-82eee792-c138-466e-a884-6e58219975fd None None] MFA Rules not processed for user `673a72ac9a9847e3812007b66ac7a5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:07.506206 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-82eee792-c138-466e-a884-6e58219975fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:07.513339 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-342677f9-23a9-4300-b6e6-43890016723b None None] MFA Rules not processed for user `35978b8fcccc4b57bb8d4a15032e708a`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:07.516998 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-342677f9-23a9-4300-b6e6-43890016723b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:07.594693 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:07.617728 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:07.689633 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 830/1662] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 19:12:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:07.690659 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eee792-c138-466e-a884-6e58219975fd None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:07.715579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 833/1663] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:07 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:07.812247 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eee792-c138-466e-a884-6e58219975fd None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:07.816368 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d481d8b-292f-4e79-9c8d-56d65b44a896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:07.817070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d481d8b-292f-4e79-9c8d-56d65b44a896 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:07.817504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d481d8b-292f-4e79-9c8d-56d65b44a896 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:07.817842 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d481d8b-292f-4e79-9c8d-56d65b44a896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:07.986096 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d481d8b-292f-4e79-9c8d-56d65b44a896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:07.987853 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d481d8b-292f-4e79-9c8d-56d65b44a896 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:08.021375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 831/1664] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:08.131747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342677f9-23a9-4300-b6e6-43890016723b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:08.145256 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342677f9-23a9-4300-b6e6-43890016723b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:08.155944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24101328-2836-4891-983a-0d90777ac8fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:08.158141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24101328-2836-4891-983a-0d90777ac8fc service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:08.158538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24101328-2836-4891-983a-0d90777ac8fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:08.159328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24101328-2836-4891-983a-0d90777ac8fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:08.188162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24101328-2836-4891-983a-0d90777ac8fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:08.192124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24101328-2836-4891-983a-0d90777ac8fc service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:08.233912 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d481d8b-292f-4e79-9c8d-56d65b44a896 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:08.236050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 834/1665] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:08.243709 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d481d8b-292f-4e79-9c8d-56d65b44a896 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:08.249273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929542f9-628e-49d4-b34d-9db9c244a87d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:08.250081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929542f9-628e-49d4-b34d-9db9c244a87d service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:08.250533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929542f9-628e-49d4-b34d-9db9c244a87d service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:08.250951 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929542f9-628e-49d4-b34d-9db9c244a87d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:08.269150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929542f9-628e-49d4-b34d-9db9c244a87d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:08.273001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929542f9-628e-49d4-b34d-9db9c244a87d service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:08.290114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 832/1666] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:08 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:12:10.970874 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24101328-2836-4891-983a-0d90777ac8fc service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:11.143542 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24101328-2836-4891-983a-0d90777ac8fc service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:11.162312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df0e691-37fb-428c-8b9b-a04fc55e9763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:11.163326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0e691-37fb-428c-8b9b-a04fc55e9763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:11.163820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0e691-37fb-428c-8b9b-a04fc55e9763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:11.164290 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df0e691-37fb-428c-8b9b-a04fc55e9763 admin admin] PATH_INFO: `/v3/users/b9b0860b541a43278543b82b9e775bbb` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:11.167882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0e691-37fb-428c-8b9b-a04fc55e9763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9b0860b541a43278543b82b9e775bbb)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:11.170287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df0e691-37fb-428c-8b9b-a04fc55e9763 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:11.240378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2df0e691-37fb-428c-8b9b-a04fc55e9763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9b0860b541a43278543b82b9e775bbb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:11.256643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2df0e691-37fb-428c-8b9b-a04fc55e9763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9b0860b541a43278543b82b9e775bbb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:11.277408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2df0e691-37fb-428c-8b9b-a04fc55e9763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9b0860b541a43278543b82b9e775bbb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:11.278089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2df0e691-37fb-428c-8b9b-a04fc55e9763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9b0860b541a43278543b82b9e775bbb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:11.283197 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929542f9-628e-49d4-b34d-9db9c244a87d service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:11.293895 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929542f9-628e-49d4-b34d-9db9c244a87d service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:11.299551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247d11c-3c0f-4cf3-9705-6432a1e968e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:11.300582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247d11c-3c0f-4cf3-9705-6432a1e968e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:11.301182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247d11c-3c0f-4cf3-9705-6432a1e968e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:11.301681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f247d11c-3c0f-4cf3-9705-6432a1e968e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:11.303652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f247d11c-3c0f-4cf3-9705-6432a1e968e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:11.309238 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2df0e691-37fb-428c-8b9b-a04fc55e9763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9b0860b541a43278543b82b9e775bbb'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:11.325466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 835/1667] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:10 2022] DELETE /identity/v3/users/b9b0860b541a43278543b82b9e775bbb => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:11.331490 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b9f80-9637-430b-bbe2-dd782c549970 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:11.331776 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b9f80-9637-430b-bbe2-dd782c549970 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:11.332052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b9f80-9637-430b-bbe2-dd782c549970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:11.386468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-398b9f80-9637-430b-bbe2-dd782c549970 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:11.387496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-398b9f80-9637-430b-bbe2-dd782c549970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:11.418045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247d11c-3c0f-4cf3-9705-6432a1e968e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:11.419894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f247d11c-3c0f-4cf3-9705-6432a1e968e0 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:11.438399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:11.449674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 833/1668] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:12:11.535599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 836/1669] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:11.577530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247d11c-3c0f-4cf3-9705-6432a1e968e0 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:11.579087 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f247d11c-3c0f-4cf3-9705-6432a1e968e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:11.714164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:11.845219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f247d11c-3c0f-4cf3-9705-6432a1e968e0 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:12.053147 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101976f-5a22-4d43-a036-60a9ab0ab9d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:12.053966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101976f-5a22-4d43-a036-60a9ab0ab9d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:12.054248 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101976f-5a22-4d43-a036-60a9ab0ab9d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:12.054520 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101976f-5a22-4d43-a036-60a9ab0ab9d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:12.074294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101976f-5a22-4d43-a036-60a9ab0ab9d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:12.075745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101976f-5a22-4d43-a036-60a9ab0ab9d3 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:12.123462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 834/1670] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:13.079454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b9f80-9637-430b-bbe2-dd782c549970 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:13.081466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-398b9f80-9637-430b-bbe2-dd782c549970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:13.107303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b9f80-9637-430b-bbe2-dd782c549970 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:13.127079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67763d13-fbf6-4670-ac7f-1b1319fad1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:13.127781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67763d13-fbf6-4670-ac7f-1b1319fad1e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:13.128247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67763d13-fbf6-4670-ac7f-1b1319fad1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:13.128533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67763d13-fbf6-4670-ac7f-1b1319fad1e6 admin admin] PATH_INFO: `/v3/projects/1b4974b4192c453e8824af0c0b2aaaab` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:13.133710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67763d13-fbf6-4670-ac7f-1b1319fad1e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b4974b4192c453e8824af0c0b2aaaab)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:13.134766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: WARNING py.warnings [None req-67763d13-fbf6-4670-ac7f-1b1319fad1e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:12:13.134766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: warnings.warn(msg) Aug 31 19:12:13.134766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: Aug 31 19:12:13.136027 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67763d13-fbf6-4670-ac7f-1b1319fad1e6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:13.312871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-67763d13-fbf6-4670-ac7f-1b1319fad1e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b4974b4192c453e8824af0c0b2aaaab', 'request_id': 'req-67763d13-fbf6-4670-ac7f-1b1319fad1e6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:13.350347 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-67763d13-fbf6-4670-ac7f-1b1319fad1e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b4974b4192c453e8824af0c0b2aaaab', 'request_id': 'req-67763d13-fbf6-4670-ac7f-1b1319fad1e6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:13.361067 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-67763d13-fbf6-4670-ac7f-1b1319fad1e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b4974b4192c453e8824af0c0b2aaaab', 'request_id': 'req-67763d13-fbf6-4670-ac7f-1b1319fad1e6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:13.383999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-67763d13-fbf6-4670-ac7f-1b1319fad1e6 admin admin] The token cache is being invalidate because project 1b4974b4192c453e8824af0c0b2aaaab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:13.383999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-67763d13-fbf6-4670-ac7f-1b1319fad1e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:13.384394 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-67763d13-fbf6-4670-ac7f-1b1319fad1e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:13.387470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 837/1671] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:13 2022] DELETE /identity/v3/projects/1b4974b4192c453e8824af0c0b2aaaab => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:13.396213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b0619-7ab7-46a7-ad55-8f6481426a87 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:13.396809 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b0619-7ab7-46a7-ad55-8f6481426a87 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:13.396966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15b0619-7ab7-46a7-ad55-8f6481426a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:13.451061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b15b0619-7ab7-46a7-ad55-8f6481426a87 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:13.453146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b15b0619-7ab7-46a7-ad55-8f6481426a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:13.618767 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 835/1672] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:13.627493 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21850ae9-a38b-40a9-a432-53579b021719 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:13.628402 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21850ae9-a38b-40a9-a432-53579b021719 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:13.629152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21850ae9-a38b-40a9-a432-53579b021719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:13.677685 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-21850ae9-a38b-40a9-a432-53579b021719 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:13.680216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-21850ae9-a38b-40a9-a432-53579b021719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:13.712644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 838/1673] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:13.717502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b0619-7ab7-46a7-ad55-8f6481426a87 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:13.730763 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15b0619-7ab7-46a7-ad55-8f6481426a87 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:13.752194 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a603475-2275-4928-9eaf-22472ee916f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:13.753064 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a603475-2275-4928-9eaf-22472ee916f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:13.753671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a603475-2275-4928-9eaf-22472ee916f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:13.754086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a603475-2275-4928-9eaf-22472ee916f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:13.754909 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a603475-2275-4928-9eaf-22472ee916f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:13.756048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a603475-2275-4928-9eaf-22472ee916f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:13.763003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a603475-2275-4928-9eaf-22472ee916f0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:13.795539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 836/1674] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:12:13.800588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21850ae9-a38b-40a9-a432-53579b021719 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:13.811757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21850ae9-a38b-40a9-a432-53579b021719 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:13.818824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363942ea-077b-491d-954a-980912a1bf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:13.821884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363942ea-077b-491d-954a-980912a1bf44 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:13.823951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363942ea-077b-491d-954a-980912a1bf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:13.824408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363942ea-077b-491d-954a-980912a1bf44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:13.825268 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363942ea-077b-491d-954a-980912a1bf44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:13.827699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363942ea-077b-491d-954a-980912a1bf44 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:13.949587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a603475-2275-4928-9eaf-22472ee916f0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:13.951061 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 839/1675] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:13 2022] POST /identity/v3/projects => generated 371 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:13.972237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a603475-2275-4928-9eaf-22472ee916f0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:14.144279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a624048-6778-4c87-8735-03e396b21416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:14.144922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a624048-6778-4c87-8735-03e396b21416 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:14.146322 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a624048-6778-4c87-8735-03e396b21416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:14.146667 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a624048-6778-4c87-8735-03e396b21416 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:14.148656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a624048-6778-4c87-8735-03e396b21416 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:14.154079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a624048-6778-4c87-8735-03e396b21416 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:14.174397 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:14.209819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363942ea-077b-491d-954a-980912a1bf44 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:14.221087 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 837/1676] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:13 2022] POST /identity/v3/users => generated 286 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:14.230030 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363942ea-077b-491d-954a-980912a1bf44 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:14.237531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae0163-c933-4b5e-80eb-41d7c6787bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:14.238163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae0163-c933-4b5e-80eb-41d7c6787bd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:14.238443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae0163-c933-4b5e-80eb-41d7c6787bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:14.238712 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae0163-c933-4b5e-80eb-41d7c6787bd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:14.239490 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae0163-c933-4b5e-80eb-41d7c6787bd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:14.243439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae0163-c933-4b5e-80eb-41d7c6787bd6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:14.268775 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a624048-6778-4c87-8735-03e396b21416 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:14.277570 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 840/1677] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:14.282276 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a624048-6778-4c87-8735-03e396b21416 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:14.288553 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f403cd-1ed1-44cd-a660-191c9c9c4aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:14.288932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f403cd-1ed1-44cd-a660-191c9c9c4aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:14.289292 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f403cd-1ed1-44cd-a660-191c9c9c4aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:14.289576 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f403cd-1ed1-44cd-a660-191c9c9c4aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:14.290269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f403cd-1ed1-44cd-a660-191c9c9c4aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:14.293935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f403cd-1ed1-44cd-a660-191c9c9c4aa7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:14.315349 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae0163-c933-4b5e-80eb-41d7c6787bd6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:14.331364 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 838/1678] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:14.333081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa4df1-07df-4238-aa81-4d94477a727d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:14.333535 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae0163-c933-4b5e-80eb-41d7c6787bd6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:14.335385 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa4df1-07df-4238-aa81-4d94477a727d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:14.337092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9aa4df1-07df-4238-aa81-4d94477a727d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:14.350616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25cbbe-5baa-42f7-bc94-0d4f0535295d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:14.351335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25cbbe-5baa-42f7-bc94-0d4f0535295d admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:14.351618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25cbbe-5baa-42f7-bc94-0d4f0535295d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:14.351894 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25cbbe-5baa-42f7-bc94-0d4f0535295d admin admin] PATH_INFO: `/v3/projects/9d8b9038e1c54375bf60744b9d41196f/users/6d7a54c97fae4af49d1f77ed0c5a6e3a/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:14.394644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25cbbe-5baa-42f7-bc94-0d4f0535295d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d8b9038e1c54375bf60744b9d41196f, user_id=6d7a54c97fae4af49d1f77ed0c5a6e3a, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:14.403170 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25cbbe-5baa-42f7-bc94-0d4f0535295d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:14.406158 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c9aa4df1-07df-4238-aa81-4d94477a727d None None] MFA Rules not processed for user `aa81a686e4324370a20e3fc35e5cc7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:14.408341 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c9aa4df1-07df-4238-aa81-4d94477a727d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:14.431570 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 841/1679] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:14 2022] PUT /identity/v3/projects/9d8b9038e1c54375bf60744b9d41196f/users/6d7a54c97fae4af49d1f77ed0c5a6e3a/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:12:14.448388 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c379bcf-b932-4405-9de3-e0eaefb774e6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:14.448674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c379bcf-b932-4405-9de3-e0eaefb774e6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:14.448955 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c379bcf-b932-4405-9de3-e0eaefb774e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:14.529657 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-5c379bcf-b932-4405-9de3-e0eaefb774e6 None None] MFA Rules not processed for user `6d7a54c97fae4af49d1f77ed0c5a6e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:14.532447 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5c379bcf-b932-4405-9de3-e0eaefb774e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:14.578182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 839/1680] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:14.590115 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc1ad0-bfe0-4915-b1f2-d68b4b1b718c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:14.590399 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc1ad0-bfe0-4915-b1f2-d68b4b1b718c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:14.590673 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc1ad0-bfe0-4915-b1f2-d68b4b1b718c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:14.650252 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e3cc1ad0-bfe0-4915-b1f2-d68b4b1b718c None None] MFA Rules not processed for user `aa81a686e4324370a20e3fc35e5cc7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:14.652076 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e3cc1ad0-bfe0-4915-b1f2-d68b4b1b718c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:14.664649 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:14.747700 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 842/1681] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:14.759330 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c379bcf-b932-4405-9de3-e0eaefb774e6 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:14.764865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5c379bcf-b932-4405-9de3-e0eaefb774e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:14.823964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 840/1682] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:12:14.841479 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc1ad0-bfe0-4915-b1f2-d68b4b1b718c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:14.843068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e3cc1ad0-bfe0-4915-b1f2-d68b4b1b718c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:14.936387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c379bcf-b932-4405-9de3-e0eaefb774e6 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:14.950997 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46616d7-3af5-4fc1-b85c-a1c7b0c570f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:14.951953 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46616d7-3af5-4fc1-b85c-a1c7b0c570f9 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:14.952301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46616d7-3af5-4fc1-b85c-a1c7b0c570f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:14.952639 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46616d7-3af5-4fc1-b85c-a1c7b0c570f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:14.978152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46616d7-3af5-4fc1-b85c-a1c7b0c570f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:14.979881 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46616d7-3af5-4fc1-b85c-a1c7b0c570f9 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:15.011873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 843/1683] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:15.082293 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc1ad0-bfe0-4915-b1f2-d68b4b1b718c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:15.088919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706dabb4-56cb-47d8-8fb9-388b1cc605d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:15.089643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dabb4-56cb-47d8-8fb9-388b1cc605d0 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:15.089841 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dabb4-56cb-47d8-8fb9-388b1cc605d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:15.090118 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706dabb4-56cb-47d8-8fb9-388b1cc605d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:15.111946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706dabb4-56cb-47d8-8fb9-388b1cc605d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:15.118981 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706dabb4-56cb-47d8-8fb9-388b1cc605d0 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:15.132664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 841/1684] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:12:15.508507 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46616d7-3af5-4fc1-b85c-a1c7b0c570f9 service nova] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:15.649718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46616d7-3af5-4fc1-b85c-a1c7b0c570f9 service nova] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:15.660750 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632b2dc-2a5f-49b7-8eea-f66a2c46d518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:15.661511 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632b2dc-2a5f-49b7-8eea-f66a2c46d518 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:15.661860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632b2dc-2a5f-49b7-8eea-f66a2c46d518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:15.662208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2632b2dc-2a5f-49b7-8eea-f66a2c46d518 admin admin] PATH_INFO: `/v3/users/6d7a54c97fae4af49d1f77ed0c5a6e3a` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:15.664085 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632b2dc-2a5f-49b7-8eea-f66a2c46d518 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d7a54c97fae4af49d1f77ed0c5a6e3a)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:15.666114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2632b2dc-2a5f-49b7-8eea-f66a2c46d518 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:15.782613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2632b2dc-2a5f-49b7-8eea-f66a2c46d518 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d7a54c97fae4af49d1f77ed0c5a6e3a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:15.792166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2632b2dc-2a5f-49b7-8eea-f66a2c46d518 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d7a54c97fae4af49d1f77ed0c5a6e3a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:15.804150 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2632b2dc-2a5f-49b7-8eea-f66a2c46d518 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d7a54c97fae4af49d1f77ed0c5a6e3a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:15.805677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2632b2dc-2a5f-49b7-8eea-f66a2c46d518 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d7a54c97fae4af49d1f77ed0c5a6e3a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:15.809911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:15.827106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2632b2dc-2a5f-49b7-8eea-f66a2c46d518 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d7a54c97fae4af49d1f77ed0c5a6e3a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:15.839679 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 844/1685] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:15 2022] DELETE /identity/v3/users/6d7a54c97fae4af49d1f77ed0c5a6e3a => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:15.844955 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cb191-0342-4fbe-93d9-7faec9b0ed13 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:15.845465 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cb191-0342-4fbe-93d9-7faec9b0ed13 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:15.845745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8cb191-0342-4fbe-93d9-7faec9b0ed13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:15.861147 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:15.887469 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-fe8cb191-0342-4fbe-93d9-7faec9b0ed13 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:15.891477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-fe8cb191-0342-4fbe-93d9-7faec9b0ed13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:15.974159 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 842/1686] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:15.997715 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632b2dc-2a5f-49b7-8eea-f66a2c46d518 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:15.999596 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2632b2dc-2a5f-49b7-8eea-f66a2c46d518 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:16.149798 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2632b2dc-2a5f-49b7-8eea-f66a2c46d518 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:16.154822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84f1857-abc6-478b-b754-d1114fba28ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:16.155450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f1857-abc6-478b-b754-d1114fba28ad service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:16.155727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f1857-abc6-478b-b754-d1114fba28ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:16.156001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84f1857-abc6-478b-b754-d1114fba28ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:16.172905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84f1857-abc6-478b-b754-d1114fba28ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:16.176304 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84f1857-abc6-478b-b754-d1114fba28ad service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:16.193949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 845/1687] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:16.575960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509aff2c-7716-4be2-bdbd-3a36498e226b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:16.576252 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509aff2c-7716-4be2-bdbd-3a36498e226b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:16.576640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509aff2c-7716-4be2-bdbd-3a36498e226b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:16.634562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-509aff2c-7716-4be2-bdbd-3a36498e226b None None] MFA Rules not processed for user `0bfa1533b0944bd681f4c2a33cf701b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:16.636867 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-509aff2c-7716-4be2-bdbd-3a36498e226b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:16.662060 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7f02f-6c35-4492-9df9-95c6ef1b1ac9 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:16.662973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7f02f-6c35-4492-9df9-95c6ef1b1ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:16.663501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7f02f-6c35-4492-9df9-95c6ef1b1ac9 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:16.665477 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 846/1688] 10.222.0.65 () {54 vars in 912 bytes} [Wed Aug 31 19:12:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:16.673789 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344eafc-fe9c-43d7-8ef9-3775d96b0925 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:16.674543 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344eafc-fe9c-43d7-8ef9-3775d96b0925 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:16.674983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344eafc-fe9c-43d7-8ef9-3775d96b0925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:16.761715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 843/1689] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:16.767997 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509aff2c-7716-4be2-bdbd-3a36498e226b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:16.770437 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-509aff2c-7716-4be2-bdbd-3a36498e226b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:16.798869 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f344eafc-fe9c-43d7-8ef9-3775d96b0925 None None] MFA Rules not processed for user `093edc8c192c469f9532a45bb3326612`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:16.800902 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f344eafc-fe9c-43d7-8ef9-3775d96b0925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:16.853721 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 847/1690] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 19:12:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:16.957888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509aff2c-7716-4be2-bdbd-3a36498e226b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:16.961287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344eafc-fe9c-43d7-8ef9-3775d96b0925 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:16.974423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344eafc-fe9c-43d7-8ef9-3775d96b0925 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:16.975472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6fb57-d02d-45ec-9050-ccd35f6d673c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:16.976086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6fb57-d02d-45ec-9050-ccd35f6d673c service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:16.976367 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6fb57-d02d-45ec-9050-ccd35f6d673c service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:16.976642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d6fb57-d02d-45ec-9050-ccd35f6d673c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:16.980227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff75fda-d9f6-457f-9049-212bb27a1d1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:16.981333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff75fda-d9f6-457f-9049-212bb27a1d1c service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:16.982998 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff75fda-d9f6-457f-9049-212bb27a1d1c service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:16.983489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff75fda-d9f6-457f-9049-212bb27a1d1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:16.994272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff75fda-d9f6-457f-9049-212bb27a1d1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:16.998227 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6fb57-d02d-45ec-9050-ccd35f6d673c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:17.000470 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d6fb57-d02d-45ec-9050-ccd35f6d673c service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:17.002736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff75fda-d9f6-457f-9049-212bb27a1d1c service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:17.013447 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 848/1691] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:17.021644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 844/1692] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:18.064174 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff75fda-d9f6-457f-9049-212bb27a1d1c service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:18.066792 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cff75fda-d9f6-457f-9049-212bb27a1d1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:18.154820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff75fda-d9f6-457f-9049-212bb27a1d1c service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:18.171703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a43e03-a565-4ea3-9b72-1b7d26c7abdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:18.173036 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a43e03-a565-4ea3-9b72-1b7d26c7abdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:18.173484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a43e03-a565-4ea3-9b72-1b7d26c7abdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:18.173884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a43e03-a565-4ea3-9b72-1b7d26c7abdf admin admin] PATH_INFO: `/v3/projects/9d8b9038e1c54375bf60744b9d41196f` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:18.177314 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a43e03-a565-4ea3-9b72-1b7d26c7abdf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d8b9038e1c54375bf60744b9d41196f)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:18.184159 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a43e03-a565-4ea3-9b72-1b7d26c7abdf admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:18.297383 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6fb57-d02d-45ec-9050-ccd35f6d673c service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:18.300566 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-51d6fb57-d02d-45ec-9050-ccd35f6d673c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:18.341478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d6fb57-d02d-45ec-9050-ccd35f6d673c service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:18.387853 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:18.409212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d7a43e03-a565-4ea3-9b72-1b7d26c7abdf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d8b9038e1c54375bf60744b9d41196f', 'request_id': 'req-d7a43e03-a565-4ea3-9b72-1b7d26c7abdf'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:18.428177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d7a43e03-a565-4ea3-9b72-1b7d26c7abdf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d8b9038e1c54375bf60744b9d41196f', 'request_id': 'req-d7a43e03-a565-4ea3-9b72-1b7d26c7abdf'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:18.445838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d7a43e03-a565-4ea3-9b72-1b7d26c7abdf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d8b9038e1c54375bf60744b9d41196f', 'request_id': 'req-d7a43e03-a565-4ea3-9b72-1b7d26c7abdf'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:18.467312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d7a43e03-a565-4ea3-9b72-1b7d26c7abdf admin admin] The token cache is being invalidate because project 9d8b9038e1c54375bf60744b9d41196f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:18.467599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d7a43e03-a565-4ea3-9b72-1b7d26c7abdf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:18.469545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d7a43e03-a565-4ea3-9b72-1b7d26c7abdf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:18.475502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 849/1693] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:18 2022] DELETE /identity/v3/projects/9d8b9038e1c54375bf60744b9d41196f => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:18.476926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff4182-2fe5-4922-ade6-9b887a0d6339 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:18.477227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff4182-2fe5-4922-ade6-9b887a0d6339 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:18.477505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff4182-2fe5-4922-ade6-9b887a0d6339 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:18.481133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 850/1694] 10.222.0.65 () {54 vars in 961 bytes} [Wed Aug 31 19:12:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:18.482194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e6292-2317-48c5-b59a-9b165230646e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:18.482472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e6292-2317-48c5-b59a-9b165230646e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:18.482756 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17e6292-2317-48c5-b59a-9b165230646e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:18.535937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c17e6292-2317-48c5-b59a-9b165230646e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:18.537165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c17e6292-2317-48c5-b59a-9b165230646e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:18.571289 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e025b32-c5ba-46ae-b523-53939621bcce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:18.572176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e025b32-c5ba-46ae-b523-53939621bcce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:18.572662 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e025b32-c5ba-46ae-b523-53939621bcce admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:18.573090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e025b32-c5ba-46ae-b523-53939621bcce admin admin] PATH_INFO: `/v3/users/9b25c27fd33d4e828b234da80a1449c2` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:18.575065 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e025b32-c5ba-46ae-b523-53939621bcce admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b25c27fd33d4e828b234da80a1449c2)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:18.576522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: WARNING py.warnings [None req-3e025b32-c5ba-46ae-b523-53939621bcce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 31 19:12:18.576522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: warnings.warn(msg) Aug 31 19:12:18.576522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: Aug 31 19:12:18.583736 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e025b32-c5ba-46ae-b523-53939621bcce admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:18.618598 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:18.690300 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 851/1695] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:18.691658 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350227a4-b625-46d4-90f0-adac9a80d080 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:18.691950 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350227a4-b625-46d4-90f0-adac9a80d080 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:18.692218 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350227a4-b625-46d4-90f0-adac9a80d080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:18.729237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3e025b32-c5ba-46ae-b523-53939621bcce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b25c27fd33d4e828b234da80a1449c2'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:18.738733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-350227a4-b625-46d4-90f0-adac9a80d080 None None] MFA Rules not processed for user `74f1846e395e4d5483fcbb41eb28a77d`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:18.740040 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-350227a4-b625-46d4-90f0-adac9a80d080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:18.747279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3e025b32-c5ba-46ae-b523-53939621bcce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b25c27fd33d4e828b234da80a1449c2'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:18.798584 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3e025b32-c5ba-46ae-b523-53939621bcce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b25c27fd33d4e828b234da80a1449c2'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:18.800097 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3e025b32-c5ba-46ae-b523-53939621bcce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b25c27fd33d4e828b234da80a1449c2'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:18.801589 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3e025b32-c5ba-46ae-b523-53939621bcce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b25c27fd33d4e828b234da80a1449c2'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:18.827383 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 845/1696] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:18 2022] DELETE /identity/v3/users/9b25c27fd33d4e828b234da80a1449c2 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:18.828758 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb309f-2cd7-4a56-b3f0-bdff41bd6a82 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:18.829222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb309f-2cd7-4a56-b3f0-bdff41bd6a82 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:18.830359 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb309f-2cd7-4a56-b3f0-bdff41bd6a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:18.891869 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 852/1697] 10.222.0.65 () {58 vars in 1058 bytes} [Wed Aug 31 19:12:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:18.893354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89b5e4-996d-4f87-b209-f51029313094 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:18.895976 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89b5e4-996d-4f87-b209-f51029313094 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:18.896153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89b5e4-996d-4f87-b209-f51029313094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:18.924260 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-60fb309f-2cd7-4a56-b3f0-bdff41bd6a82 None None] MFA Rules not processed for user `aa81a686e4324370a20e3fc35e5cc7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:18.929065 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-60fb309f-2cd7-4a56-b3f0-bdff41bd6a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:18.955989 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-1c89b5e4-996d-4f87-b209-f51029313094 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:18.963890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1c89b5e4-996d-4f87-b209-f51029313094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:19.091105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 853/1698] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:19.094412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53caaf55-036f-4c53-a460-22da4744bc74 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:19.095474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53caaf55-036f-4c53-a460-22da4744bc74 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:19.096003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53caaf55-036f-4c53-a460-22da4744bc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:19.132463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 846/1699] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:18 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:19.137012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb309f-2cd7-4a56-b3f0-bdff41bd6a82 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:19.139729 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-53caaf55-036f-4c53-a460-22da4744bc74 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:19.141749 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-53caaf55-036f-4c53-a460-22da4744bc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:19.169319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 854/1700] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:19.171056 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267b71e-13cb-47eb-a86c-c815002c7031 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:19.171481 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267b71e-13cb-47eb-a86c-c815002c7031 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:19.171910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0267b71e-13cb-47eb-a86c-c815002c7031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:19.231458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0267b71e-13cb-47eb-a86c-c815002c7031 None None] MFA Rules not processed for user `b4869c70eac84fddb30090585bc1f0de`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:19.237149 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0267b71e-13cb-47eb-a86c-c815002c7031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:19.317222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:19.364941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb309f-2cd7-4a56-b3f0-bdff41bd6a82 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:19.406234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb1aca-ed75-4f08-9353-48ef91b472fc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:19.406949 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb1aca-ed75-4f08-9353-48ef91b472fc service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:19.407186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb1aca-ed75-4f08-9353-48ef91b472fc service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:19.407512 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bb1aca-ed75-4f08-9353-48ef91b472fc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:19.410608 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f1bb1aca-ed75-4f08-9353-48ef91b472fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:19.411625 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:19.424120 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb1aca-ed75-4f08-9353-48ef91b472fc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:19.426416 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bb1aca-ed75-4f08-9353-48ef91b472fc service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:19.438372 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 855/1701] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:19.442554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 847/1702] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:19.443128 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb1aca-ed75-4f08-9353-48ef91b472fc service placement] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:19.451545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f1bb1aca-ed75-4f08-9353-48ef91b472fc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:19.451687 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267b71e-13cb-47eb-a86c-c815002c7031 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:19.482333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0267b71e-13cb-47eb-a86c-c815002c7031 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:19.506161 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981497f8-1870-4990-bf66-c0261a002923 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:19.507762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981497f8-1870-4990-bf66-c0261a002923 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:19.508273 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981497f8-1870-4990-bf66-c0261a002923 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:19.509577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981497f8-1870-4990-bf66-c0261a002923 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:19.511097 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981497f8-1870-4990-bf66-c0261a002923 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:19.511623 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981497f8-1870-4990-bf66-c0261a002923 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:19.514781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981497f8-1870-4990-bf66-c0261a002923 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:19.539197 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bb1aca-ed75-4f08-9353-48ef91b472fc service placement] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:19.549913 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41024dca-5bc4-44fe-8428-1285b555362f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:19.550923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41024dca-5bc4-44fe-8428-1285b555362f service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:19.551239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 856/1703] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:12:19.555245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41024dca-5bc4-44fe-8428-1285b555362f service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:19.555245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41024dca-5bc4-44fe-8428-1285b555362f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:19.559371 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca511e-29b9-4954-8273-25c0061566d4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:19.566072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca511e-29b9-4954-8273-25c0061566d4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:19.566504 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ca511e-29b9-4954-8273-25c0061566d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:19.576967 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41024dca-5bc4-44fe-8428-1285b555362f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:19.578719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41024dca-5bc4-44fe-8428-1285b555362f service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:19.596181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 848/1704] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:19.597036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41024dca-5bc4-44fe-8428-1285b555362f service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:19.615223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41024dca-5bc4-44fe-8428-1285b555362f service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:19.630252 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12a75c-e50a-416b-882c-aae33e6f18e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:19.630839 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12a75c-e50a-416b-882c-aae33e6f18e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:19.631099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-23ca511e-29b9-4954-8273-25c0061566d4 None None] MFA Rules not processed for user `b4869c70eac84fddb30090585bc1f0de`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:19.631210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12a75c-e50a-416b-882c-aae33e6f18e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:19.631392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d12a75c-e50a-416b-882c-aae33e6f18e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:19.632011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d12a75c-e50a-416b-882c-aae33e6f18e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:19.639297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d12a75c-e50a-416b-882c-aae33e6f18e7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:19.640981 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-23ca511e-29b9-4954-8273-25c0061566d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:19.700369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 849/1705] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:19 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:19.701330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12a75c-e50a-416b-882c-aae33e6f18e7 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:19.712839 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 857/1706] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:19.714247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca511e-29b9-4954-8273-25c0061566d4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:19.722041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-23ca511e-29b9-4954-8273-25c0061566d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:19.812616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d12a75c-e50a-416b-882c-aae33e6f18e7 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:19.825500 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5638c-d227-4cc1-a10a-63767dad7860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:19.826137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5638c-d227-4cc1-a10a-63767dad7860 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:19.826422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5638c-d227-4cc1-a10a-63767dad7860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:19.826696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da5638c-d227-4cc1-a10a-63767dad7860 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:19.827478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5638c-d227-4cc1-a10a-63767dad7860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:19.829449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da5638c-d227-4cc1-a10a-63767dad7860 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:19.886342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 850/1707] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:19 2022] POST /identity/v3/users => generated 283 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:19.890261 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5638c-d227-4cc1-a10a-63767dad7860 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:19.914031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da5638c-d227-4cc1-a10a-63767dad7860 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:19.929569 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6902f39-4962-4d21-aa78-3546861ccecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:19.930428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6902f39-4962-4d21-aa78-3546861ccecb admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:19.930893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6902f39-4962-4d21-aa78-3546861ccecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:19.931229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6902f39-4962-4d21-aa78-3546861ccecb admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:19.933795 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6902f39-4962-4d21-aa78-3546861ccecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:19.935715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6902f39-4962-4d21-aa78-3546861ccecb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:19.945220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ca511e-29b9-4954-8273-25c0061566d4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:19.974925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 851/1708] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:19.979565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6902f39-4962-4d21-aa78-3546861ccecb admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:20.008847 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6902f39-4962-4d21-aa78-3546861ccecb admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:20.021917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6e7b3-2e05-4ab3-b215-2a210637baaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:20.022717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6e7b3-2e05-4ab3-b215-2a210637baaf admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:20.023316 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6e7b3-2e05-4ab3-b215-2a210637baaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:20.024414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6e7b3-2e05-4ab3-b215-2a210637baaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:20.026381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6e7b3-2e05-4ab3-b215-2a210637baaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:20.031640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b6e7b3-2e05-4ab3-b215-2a210637baaf admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:20.092084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 852/1709] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:19 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:20.100310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6e7b3-2e05-4ab3-b215-2a210637baaf admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:20.166891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6e7b3-2e05-4ab3-b215-2a210637baaf admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:20.187085 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce9ab4e-20c6-4b6f-94f5-52839f1a0613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:20.188012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9ab4e-20c6-4b6f-94f5-52839f1a0613 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:20.189457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9ab4e-20c6-4b6f-94f5-52839f1a0613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:20.190160 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce9ab4e-20c6-4b6f-94f5-52839f1a0613 admin admin] PATH_INFO: `/v3/projects/c6ef78ed6d034a1bb69cc3c71446ac8b/users/c29df84822474a41942bcc8d761fbb34/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:20.205343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39aa15-ae25-405b-9882-0b1f262206ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:20.211746 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39aa15-ae25-405b-9882-0b1f262206ce service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:20.212167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39aa15-ae25-405b-9882-0b1f262206ce service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:20.212565 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c39aa15-ae25-405b-9882-0b1f262206ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:20.233610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9ab4e-20c6-4b6f-94f5-52839f1a0613 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6ef78ed6d034a1bb69cc3c71446ac8b, user_id=c29df84822474a41942bcc8d761fbb34, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:20.243851 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce9ab4e-20c6-4b6f-94f5-52839f1a0613 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:20.248170 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:20.274268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 853/1710] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:20 2022] PUT /identity/v3/projects/c6ef78ed6d034a1bb69cc3c71446ac8b/users/c29df84822474a41942bcc8d761fbb34/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:20.288011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e3b38-f425-4cae-ab83-df8cf517009c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:20.288444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e3b38-f425-4cae-ab83-df8cf517009c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:20.288818 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2e3b38-f425-4cae-ab83-df8cf517009c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:20.406670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4e2e3b38-f425-4cae-ab83-df8cf517009c None None] MFA Rules not processed for user `c29df84822474a41942bcc8d761fbb34`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:20.412360 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4e2e3b38-f425-4cae-ab83-df8cf517009c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:20.503248 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c39aa15-ae25-405b-9882-0b1f262206ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:20.510706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c39aa15-ae25-405b-9882-0b1f262206ce service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:20.567828 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 858/1711] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 854 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:20.578099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 854/1712] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:12:20.582613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39aa15-ae25-405b-9882-0b1f262206ce service nova] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:20.614228 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c39aa15-ae25-405b-9882-0b1f262206ce service nova] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:20.633397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:20.758340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e3b38-f425-4cae-ab83-df8cf517009c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:20.760837 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4e2e3b38-f425-4cae-ab83-df8cf517009c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:20.801757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d296e-c03d-4920-b270-7d6df603e3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:20.802938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d296e-c03d-4920-b270-7d6df603e3e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:20.803703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d296e-c03d-4920-b270-7d6df603e3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:20.803703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0d296e-c03d-4920-b270-7d6df603e3e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:20.804157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d296e-c03d-4920-b270-7d6df603e3e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:20.807911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0d296e-c03d-4920-b270-7d6df603e3e2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:20.900349 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 859/1713] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:20 2022] POST /identity/v3/projects => generated 369 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:20.905562 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d296e-c03d-4920-b270-7d6df603e3e2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:21.008201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2e3b38-f425-4cae-ab83-df8cf517009c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:21.092733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0d296e-c03d-4920-b270-7d6df603e3e2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:21.106271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9f7f5-5761-4d69-aaf3-b35c14b6ca45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:21.107004 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9f7f5-5761-4d69-aaf3-b35c14b6ca45 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:21.107316 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9f7f5-5761-4d69-aaf3-b35c14b6ca45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:21.107555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec9f7f5-5761-4d69-aaf3-b35c14b6ca45 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:21.108568 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9f7f5-5761-4d69-aaf3-b35c14b6ca45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:21.112691 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec9f7f5-5761-4d69-aaf3-b35c14b6ca45 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:21.153880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 860/1714] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:20 2022] POST /identity/v3/users => generated 291 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:21.162340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9f7f5-5761-4d69-aaf3-b35c14b6ca45 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:21.172593 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1fa81b-82f1-4c86-92eb-da743612ad92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:21.174329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1fa81b-82f1-4c86-92eb-da743612ad92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:21.174717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1fa81b-82f1-4c86-92eb-da743612ad92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:21.175130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1fa81b-82f1-4c86-92eb-da743612ad92 admin admin] PATH_INFO: `/v3/projects/ef754536a67849f59a2f5baa3d51411c` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:21.176622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1fa81b-82f1-4c86-92eb-da743612ad92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef754536a67849f59a2f5baa3d51411c)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:21.178765 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1fa81b-82f1-4c86-92eb-da743612ad92 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:21.193927 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec9f7f5-5761-4d69-aaf3-b35c14b6ca45 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:21.206937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312eb734-5955-43af-97bd-fe6ec445b420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:21.213628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312eb734-5955-43af-97bd-fe6ec445b420 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:21.213628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312eb734-5955-43af-97bd-fe6ec445b420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:21.214642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312eb734-5955-43af-97bd-fe6ec445b420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:21.215348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312eb734-5955-43af-97bd-fe6ec445b420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:21.233948 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312eb734-5955-43af-97bd-fe6ec445b420 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:21.251822 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:21.265434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9f1fa81b-82f1-4c86-92eb-da743612ad92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef754536a67849f59a2f5baa3d51411c', 'request_id': 'req-9f1fa81b-82f1-4c86-92eb-da743612ad92'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:21.274850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 861/1715] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:21.280747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312eb734-5955-43af-97bd-fe6ec445b420 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:21.281056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9f1fa81b-82f1-4c86-92eb-da743612ad92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef754536a67849f59a2f5baa3d51411c', 'request_id': 'req-9f1fa81b-82f1-4c86-92eb-da743612ad92'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:21.290818 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9f1fa81b-82f1-4c86-92eb-da743612ad92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef754536a67849f59a2f5baa3d51411c', 'request_id': 'req-9f1fa81b-82f1-4c86-92eb-da743612ad92'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:21.308221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312eb734-5955-43af-97bd-fe6ec445b420 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:21.311128 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9f1fa81b-82f1-4c86-92eb-da743612ad92 admin admin] The token cache is being invalidate because project ef754536a67849f59a2f5baa3d51411c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:21.312856 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9f1fa81b-82f1-4c86-92eb-da743612ad92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:21.315866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9f1fa81b-82f1-4c86-92eb-da743612ad92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:21.317238 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 855/1716] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:20 2022] DELETE /identity/v3/projects/ef754536a67849f59a2f5baa3d51411c => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:21.321916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1fa81b-82f1-4c86-92eb-da743612ad92 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:21.326129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9f1fa81b-82f1-4c86-92eb-da743612ad92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:21.474309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1fa81b-82f1-4c86-92eb-da743612ad92 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:21.483696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e77cbc-6407-4f40-b9e3-f5842bf5f579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:21.485840 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e77cbc-6407-4f40-b9e3-f5842bf5f579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:21.486444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e77cbc-6407-4f40-b9e3-f5842bf5f579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:21.487103 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e77cbc-6407-4f40-b9e3-f5842bf5f579 admin admin] PATH_INFO: `/v3/users/7deaaeb48b8c40d28a24cf14db40c07c` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:21.489131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e77cbc-6407-4f40-b9e3-f5842bf5f579 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7deaaeb48b8c40d28a24cf14db40c07c)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:21.503485 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e77cbc-6407-4f40-b9e3-f5842bf5f579 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:21.530412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342b6b1-c311-449e-8cde-98a7cba5929a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:21.533711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342b6b1-c311-449e-8cde-98a7cba5929a admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:21.537631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342b6b1-c311-449e-8cde-98a7cba5929a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:21.537952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b342b6b1-c311-449e-8cde-98a7cba5929a admin admin] PATH_INFO: `/v3/projects/e1087d1c9a9f4628bd7c3ea53958a3c9/users/0dff4a0a1ba14906b7d9e9481b34ca4e/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:21.596709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342b6b1-c311-449e-8cde-98a7cba5929a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1087d1c9a9f4628bd7c3ea53958a3c9, user_id=0dff4a0a1ba14906b7d9e9481b34ca4e, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:21.603821 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b342b6b1-c311-449e-8cde-98a7cba5929a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:21.612686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:21.654274 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 862/1717] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:21 2022] PUT /identity/v3/projects/e1087d1c9a9f4628bd7c3ea53958a3c9/users/0dff4a0a1ba14906b7d9e9481b34ca4e/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:21.655388 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342b6b1-c311-449e-8cde-98a7cba5929a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:21.657664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b342b6b1-c311-449e-8cde-98a7cba5929a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:21.706648 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c2e77cbc-6407-4f40-b9e3-f5842bf5f579 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7deaaeb48b8c40d28a24cf14db40c07c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:21.734308 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c2e77cbc-6407-4f40-b9e3-f5842bf5f579 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7deaaeb48b8c40d28a24cf14db40c07c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:21.764971 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c2e77cbc-6407-4f40-b9e3-f5842bf5f579 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7deaaeb48b8c40d28a24cf14db40c07c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:21.766122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c2e77cbc-6407-4f40-b9e3-f5842bf5f579 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7deaaeb48b8c40d28a24cf14db40c07c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:21.766959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c2e77cbc-6407-4f40-b9e3-f5842bf5f579 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7deaaeb48b8c40d28a24cf14db40c07c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:21.798115 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 856/1718] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:21 2022] DELETE /identity/v3/users/7deaaeb48b8c40d28a24cf14db40c07c => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:21.813677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b342b6b1-c311-449e-8cde-98a7cba5929a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:21.816146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daee680-a1c9-4867-bde2-c96099eafda7 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:21.819538 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daee680-a1c9-4867-bde2-c96099eafda7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:21.819977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daee680-a1c9-4867-bde2-c96099eafda7 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:21.828265 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 857/1719] 10.222.0.65 () {54 vars in 912 bytes} [Wed Aug 31 19:12:21 2022] GET /identity => generated 270 bytes in 29 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:21.834941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22776ed-186e-461e-b445-d7facbb71bd0 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:21.835227 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22776ed-186e-461e-b445-d7facbb71bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:21.839902 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22776ed-186e-461e-b445-d7facbb71bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:21.923266 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b22776ed-186e-461e-b445-d7facbb71bd0 None None] MFA Rules not processed for user `093edc8c192c469f9532a45bb3326612`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:21.930441 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b22776ed-186e-461e-b445-d7facbb71bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:22.003236 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaf068-6b3c-4262-bf53-c2bdfbac3772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:22.003867 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaf068-6b3c-4262-bf53-c2bdfbac3772 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:22.004156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaf068-6b3c-4262-bf53-c2bdfbac3772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:22.005311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfaf068-6b3c-4262-bf53-c2bdfbac3772 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:22.006164 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaf068-6b3c-4262-bf53-c2bdfbac3772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:22.011024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfaf068-6b3c-4262-bf53-c2bdfbac3772 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:22.041922 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 863/1720] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:12:22.046928 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaf068-6b3c-4262-bf53-c2bdfbac3772 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:22.048267 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:22.050518 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5cfaf068-6b3c-4262-bf53-c2bdfbac3772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:22.083927 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfaf068-6b3c-4262-bf53-c2bdfbac3772 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:22.090621 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d74c230-3f5f-4360-b672-80ee7ad01ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:22.091477 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74c230-3f5f-4360-b672-80ee7ad01ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:22.092687 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74c230-3f5f-4360-b672-80ee7ad01ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:22.093335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d74c230-3f5f-4360-b672-80ee7ad01ef0 admin admin] PATH_INFO: `/v3/projects/e1087d1c9a9f4628bd7c3ea53958a3c9/users/0dff4a0a1ba14906b7d9e9481b34ca4e/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:22.095939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74c230-3f5f-4360-b672-80ee7ad01ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1087d1c9a9f4628bd7c3ea53958a3c9, user_id=0dff4a0a1ba14906b7d9e9481b34ca4e, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:22.098980 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d74c230-3f5f-4360-b672-80ee7ad01ef0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:22.115563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 864/1721] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:22 2022] PUT /identity/v3/projects/e1087d1c9a9f4628bd7c3ea53958a3c9/users/0dff4a0a1ba14906b7d9e9481b34ca4e/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:22.133722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180a586-7656-4d0d-a9cd-d9680cfcd9a2 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:22.135008 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180a586-7656-4d0d-a9cd-d9680cfcd9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:22.139871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 858/1722] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 19:12:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:22.142875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7180a586-7656-4d0d-a9cd-d9680cfcd9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:22.253653 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7180a586-7656-4d0d-a9cd-d9680cfcd9a2 None None] MFA Rules not processed for user `0dff4a0a1ba14906b7d9e9481b34ca4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:22.255050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7180a586-7656-4d0d-a9cd-d9680cfcd9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:22.259627 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22776ed-186e-461e-b445-d7facbb71bd0 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:22.261296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b22776ed-186e-461e-b445-d7facbb71bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:22.293878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22776ed-186e-461e-b445-d7facbb71bd0 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:22.473370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 865/1723] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:22 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:22.475529 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202eaf94-141f-4605-9bf4-5185d0bec4b8 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:22.476128 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202eaf94-141f-4605-9bf4-5185d0bec4b8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:22.476646 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202eaf94-141f-4605-9bf4-5185d0bec4b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:22.494415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9d308-ed57-4434-97f2-2ff2a2893bb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:22.495791 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9d308-ed57-4434-97f2-2ff2a2893bb9 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:22.496452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9d308-ed57-4434-97f2-2ff2a2893bb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:22.497118 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e9d308-ed57-4434-97f2-2ff2a2893bb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:22.509791 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e9d308-ed57-4434-97f2-2ff2a2893bb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:22.512277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e9d308-ed57-4434-97f2-2ff2a2893bb9 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:22.532712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 859/1724] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:22.536273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9d308-ed57-4434-97f2-2ff2a2893bb9 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:22.539058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-55e9d308-ed57-4434-97f2-2ff2a2893bb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:22.541607 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-202eaf94-141f-4605-9bf4-5185d0bec4b8 None None] MFA Rules not processed for user `0bfa1533b0944bd681f4c2a33cf701b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:22.542842 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-202eaf94-141f-4605-9bf4-5185d0bec4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:22.553438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:22.642153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 866/1725] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:22.720451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e9d308-ed57-4434-97f2-2ff2a2893bb9 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:22.725398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06147e-af91-4973-954d-cc5fccc6b31d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:22.726035 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06147e-af91-4973-954d-cc5fccc6b31d service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:22.726508 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06147e-af91-4973-954d-cc5fccc6b31d service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:22.726627 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06147e-af91-4973-954d-cc5fccc6b31d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:22.766810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202eaf94-141f-4605-9bf4-5185d0bec4b8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:22.769366 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-202eaf94-141f-4605-9bf4-5185d0bec4b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:22.772738 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06147e-af91-4973-954d-cc5fccc6b31d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:22.774249 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06147e-af91-4973-954d-cc5fccc6b31d service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:22.799636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 860/1726] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:22 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:22.958486 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202eaf94-141f-4605-9bf4-5185d0bec4b8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:22.966635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240e2f9-12f3-4823-a417-be1a22297d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:22.967820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240e2f9-12f3-4823-a417-be1a22297d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:22.968150 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240e2f9-12f3-4823-a417-be1a22297d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:22.968717 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240e2f9-12f3-4823-a417-be1a22297d67 admin admin] PATH_INFO: `/v3/projects/65b58bb980cc49b0a0adfbad99ae4326` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:22.971094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8240e2f9-12f3-4823-a417-be1a22297d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65b58bb980cc49b0a0adfbad99ae4326)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:22.974176 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8240e2f9-12f3-4823-a417-be1a22297d67 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:23.050344 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8240e2f9-12f3-4823-a417-be1a22297d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65b58bb980cc49b0a0adfbad99ae4326', 'request_id': 'req-8240e2f9-12f3-4823-a417-be1a22297d67'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:23.056539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8240e2f9-12f3-4823-a417-be1a22297d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65b58bb980cc49b0a0adfbad99ae4326', 'request_id': 'req-8240e2f9-12f3-4823-a417-be1a22297d67'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:23.066999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8240e2f9-12f3-4823-a417-be1a22297d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65b58bb980cc49b0a0adfbad99ae4326', 'request_id': 'req-8240e2f9-12f3-4823-a417-be1a22297d67'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:23.090763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8240e2f9-12f3-4823-a417-be1a22297d67 admin admin] The token cache is being invalidate because project 65b58bb980cc49b0a0adfbad99ae4326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:23.092416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8240e2f9-12f3-4823-a417-be1a22297d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:23.094219 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-8240e2f9-12f3-4823-a417-be1a22297d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:23.099769 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 867/1727] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:22 2022] DELETE /identity/v3/projects/65b58bb980cc49b0a0adfbad99ae4326 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:12:23.106923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba229b8-91b3-4e80-a378-a927b1fe52b3 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:23.107492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba229b8-91b3-4e80-a378-a927b1fe52b3 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:23.109281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba229b8-91b3-4e80-a378-a927b1fe52b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:23.186487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-5ba229b8-91b3-4e80-a378-a927b1fe52b3 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:23.190710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5ba229b8-91b3-4e80-a378-a927b1fe52b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:23.373694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 861/1728] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:23.376390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477baaea-0300-431b-918c-348252bb09b3 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:23.376886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477baaea-0300-431b-918c-348252bb09b3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:23.376979 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477baaea-0300-431b-918c-348252bb09b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:23.449775 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-477baaea-0300-431b-918c-348252bb09b3 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:23.451514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-477baaea-0300-431b-918c-348252bb09b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:23.468606 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 868/1729] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:23.470369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba229b8-91b3-4e80-a378-a927b1fe52b3 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:23.532246 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba229b8-91b3-4e80-a378-a927b1fe52b3 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:23.560332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dae0b6-37db-48d0-8c8e-9c8f79c1ebd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:23.560332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dae0b6-37db-48d0-8c8e-9c8f79c1ebd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:23.563901 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dae0b6-37db-48d0-8c8e-9c8f79c1ebd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:23.564171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dae0b6-37db-48d0-8c8e-9c8f79c1ebd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:23.569182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dae0b6-37db-48d0-8c8e-9c8f79c1ebd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:23.574710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dae0b6-37db-48d0-8c8e-9c8f79c1ebd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:23.585558 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dae0b6-37db-48d0-8c8e-9c8f79c1ebd2 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:23.649046 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477baaea-0300-431b-918c-348252bb09b3 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:23.659446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 862/1730] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:12:23.684943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477baaea-0300-431b-918c-348252bb09b3 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:23.723398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4ed4f-2774-4667-bb7d-02abe049629b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:23.724303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4ed4f-2774-4667-bb7d-02abe049629b admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:23.724718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4ed4f-2774-4667-bb7d-02abe049629b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:23.727951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec4ed4f-2774-4667-bb7d-02abe049629b admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:23.728738 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4ed4f-2774-4667-bb7d-02abe049629b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:23.740035 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec4ed4f-2774-4667-bb7d-02abe049629b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:23.811001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dae0b6-37db-48d0-8c8e-9c8f79c1ebd2 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:23.818919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 869/1731] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:23 2022] POST /identity/v3/projects => generated 377 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:12:23.830523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dae0b6-37db-48d0-8c8e-9c8f79c1ebd2 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:23.899807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:23.977806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8f82ec-a4ac-4c3a-926d-60758b56a2c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:23.978258 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f82ec-a4ac-4c3a-926d-60758b56a2c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:23.978417 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f82ec-a4ac-4c3a-926d-60758b56a2c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:23.989782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8f82ec-a4ac-4c3a-926d-60758b56a2c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:23.992066 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f82ec-a4ac-4c3a-926d-60758b56a2c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:23.998440 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8f82ec-a4ac-4c3a-926d-60758b56a2c0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:24.079947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 863/1732] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:23 2022] POST /identity/v3/users => generated 289 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:24.084156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4ed4f-2774-4667-bb7d-02abe049629b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:24.148128 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec4ed4f-2774-4667-bb7d-02abe049629b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:24.165264 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9055d5e-166e-49d5-9144-24752ba915d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:24.167112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9055d5e-166e-49d5-9144-24752ba915d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:24.167405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9055d5e-166e-49d5-9144-24752ba915d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:24.167673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9055d5e-166e-49d5-9144-24752ba915d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:24.172899 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9055d5e-166e-49d5-9144-24752ba915d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:24.182317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9055d5e-166e-49d5-9144-24752ba915d4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:24.196587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c24283-2f57-4443-8d67-fc0276b571f7 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:24.196857 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c24283-2f57-4443-8d67-fc0276b571f7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:24.197514 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c24283-2f57-4443-8d67-fc0276b571f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:24.241751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 870/1733] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:24.251954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9055d5e-166e-49d5-9144-24752ba915d4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:24.319577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9055d5e-166e-49d5-9144-24752ba915d4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:24.343947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-34c24283-2f57-4443-8d67-fc0276b571f7 None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:24.345265 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-34c24283-2f57-4443-8d67-fc0276b571f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:24.352430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a79861-7fb8-47cc-b876-ddb9de199893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:24.354364 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a79861-7fb8-47cc-b876-ddb9de199893 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:24.360488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a79861-7fb8-47cc-b876-ddb9de199893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:24.360805 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a79861-7fb8-47cc-b876-ddb9de199893 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:24.372250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a79861-7fb8-47cc-b876-ddb9de199893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:24.378973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a79861-7fb8-47cc-b876-ddb9de199893 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:24.390107 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:24.465240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 871/1734] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:24.475707 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a79861-7fb8-47cc-b876-ddb9de199893 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:24.519602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a79861-7fb8-47cc-b876-ddb9de199893 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:24.537389 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0ce12-6c62-41e0-85f2-28c1b171cb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:24.538360 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0ce12-6c62-41e0-85f2-28c1b171cb0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:24.538942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0ce12-6c62-41e0-85f2-28c1b171cb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:24.539391 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad0ce12-6c62-41e0-85f2-28c1b171cb0a admin admin] PATH_INFO: `/v3/projects/3061d78415204980bf871fa211f55798/users/451b44b6d3eb48c9b035db2091953a62/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:24.580180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0ce12-6c62-41e0-85f2-28c1b171cb0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3061d78415204980bf871fa211f55798, user_id=451b44b6d3eb48c9b035db2091953a62, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:24.584988 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad0ce12-6c62-41e0-85f2-28c1b171cb0a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:24.604683 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 864/1735] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 19:12:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:24.647569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 872/1736] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:24 2022] PUT /identity/v3/projects/3061d78415204980bf871fa211f55798/users/451b44b6d3eb48c9b035db2091953a62/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:24.648977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcd98d-5ffa-49ea-85a2-6dbfa5746822 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:24.649436 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcd98d-5ffa-49ea-85a2-6dbfa5746822 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:24.649843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcd98d-5ffa-49ea-85a2-6dbfa5746822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:24.771469 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2ffcd98d-5ffa-49ea-85a2-6dbfa5746822 None None] MFA Rules not processed for user `451b44b6d3eb48c9b035db2091953a62`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:24.772705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2ffcd98d-5ffa-49ea-85a2-6dbfa5746822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:25.093645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0ce12-6c62-41e0-85f2-28c1b171cb0a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:25.095610 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8ad0ce12-6c62-41e0-85f2-28c1b171cb0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:25.176802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 865/1737] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 529 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:12:25.193655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcd98d-5ffa-49ea-85a2-6dbfa5746822 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:25.283182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:25.568138 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcd98d-5ffa-49ea-85a2-6dbfa5746822 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:25.586976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983ea01-866d-44a0-b991-a1c423e0496b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:25.589472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983ea01-866d-44a0-b991-a1c423e0496b admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:25.590977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983ea01-866d-44a0-b991-a1c423e0496b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:25.592690 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8983ea01-866d-44a0-b991-a1c423e0496b admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:25.597808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983ea01-866d-44a0-b991-a1c423e0496b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:25.607198 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8983ea01-866d-44a0-b991-a1c423e0496b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:25.631631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad0ce12-6c62-41e0-85f2-28c1b171cb0a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:25.664639 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c74f1-a6cd-4820-8af1-473c54ad1ac0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:25.665588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c74f1-a6cd-4820-8af1-473c54ad1ac0 service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:25.665588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c74f1-a6cd-4820-8af1-473c54ad1ac0 service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:25.665815 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13c74f1-a6cd-4820-8af1-473c54ad1ac0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:25.679796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 866/1738] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:25 2022] POST /identity/v3/projects => generated 377 bytes in 498 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:25.681323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983ea01-866d-44a0-b991-a1c423e0496b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:25.776814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c74f1-a6cd-4820-8af1-473c54ad1ac0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:25.779929 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13c74f1-a6cd-4820-8af1-473c54ad1ac0 service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:25.818279 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 873/1739] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:12:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 726 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:12:26.011728 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8983ea01-866d-44a0-b991-a1c423e0496b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:26.036766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493895b-8aa1-4a85-9e8f-f917b6e830fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:26.038717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493895b-8aa1-4a85-9e8f-f917b6e830fb admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:26.040201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493895b-8aa1-4a85-9e8f-f917b6e830fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:26.040592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493895b-8aa1-4a85-9e8f-f917b6e830fb admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:26.044543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493895b-8aa1-4a85-9e8f-f917b6e830fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:26.046306 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493895b-8aa1-4a85-9e8f-f917b6e830fb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:26.114996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 867/1740] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:25 2022] POST /identity/v3/users => generated 295 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:26.116385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c74f1-a6cd-4820-8af1-473c54ad1ac0 service swift] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:26.138141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13c74f1-a6cd-4820-8af1-473c54ad1ac0 service swift] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:26.155886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8d4ef-66b9-4379-876d-064c88a6e5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:26.157763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8d4ef-66b9-4379-876d-064c88a6e5bf admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:26.158214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8d4ef-66b9-4379-876d-064c88a6e5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:26.159065 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c8d4ef-66b9-4379-876d-064c88a6e5bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:26.167515 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8d4ef-66b9-4379-876d-064c88a6e5bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:26.187288 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c8d4ef-66b9-4379-876d-064c88a6e5bf admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:26.252293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 874/1741] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:26.257448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493895b-8aa1-4a85-9e8f-f917b6e830fb admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:26.285740 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493895b-8aa1-4a85-9e8f-f917b6e830fb admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:26.311301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e5c68-6973-41c8-beb1-968a4a735eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:26.315959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e5c68-6973-41c8-beb1-968a4a735eb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:26.321021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e5c68-6973-41c8-beb1-968a4a735eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:26.323770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244e5c68-6973-41c8-beb1-968a4a735eb3 admin admin] PATH_INFO: `/v3/projects/412834d055f14ef8ab3e335f3df414c5/users/fb2e6bc90b3a4f198ca5495a32e4faa9/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:26.417846 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e5c68-6973-41c8-beb1-968a4a735eb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=412834d055f14ef8ab3e335f3df414c5, user_id=fb2e6bc90b3a4f198ca5495a32e4faa9, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:26.435693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244e5c68-6973-41c8-beb1-968a4a735eb3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:26.488890 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 868/1742] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:26 2022] PUT /identity/v3/projects/412834d055f14ef8ab3e335f3df414c5/users/fb2e6bc90b3a4f198ca5495a32e4faa9/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:26.497204 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8d4ef-66b9-4379-876d-064c88a6e5bf admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:26.516695 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c8d4ef-66b9-4379-876d-064c88a6e5bf admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:26.680726 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:26.754343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091027c3-8adf-4c60-996f-dc4056bb85f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:26.754976 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091027c3-8adf-4c60-996f-dc4056bb85f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:26.759830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091027c3-8adf-4c60-996f-dc4056bb85f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:26.762831 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091027c3-8adf-4c60-996f-dc4056bb85f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:26.764616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091027c3-8adf-4c60-996f-dc4056bb85f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:26.768322 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091027c3-8adf-4c60-996f-dc4056bb85f5 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:26.833965 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 875/1743] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:26 2022] GET /identity/v3/roles => generated 1420 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:26.838226 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e5c68-6973-41c8-beb1-968a4a735eb3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:26.856847 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244e5c68-6973-41c8-beb1-968a4a735eb3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:26.881973 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f4d8a-2436-43fb-a1ff-9bc1befdba91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:26.882685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f4d8a-2436-43fb-a1ff-9bc1befdba91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:26.882999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f4d8a-2436-43fb-a1ff-9bc1befdba91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:26.883278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78f4d8a-2436-43fb-a1ff-9bc1befdba91 admin admin] PATH_INFO: `/v3/projects/412834d055f14ef8ab3e335f3df414c5/users/fb2e6bc90b3a4f198ca5495a32e4faa9/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:26.896535 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f4d8a-2436-43fb-a1ff-9bc1befdba91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=412834d055f14ef8ab3e335f3df414c5, user_id=fb2e6bc90b3a4f198ca5495a32e4faa9, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:26.904185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78f4d8a-2436-43fb-a1ff-9bc1befdba91 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:26.965414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 869/1744] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:26 2022] PUT /identity/v3/projects/412834d055f14ef8ab3e335f3df414c5/users/fb2e6bc90b3a4f198ca5495a32e4faa9/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:26.975716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c81c5c-a878-4ac5-bd03-14ac34790a28 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:26.976004 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c81c5c-a878-4ac5-bd03-14ac34790a28 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:26.976281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c81c5c-a878-4ac5-bd03-14ac34790a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:27.109709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-65c81c5c-a878-4ac5-bd03-14ac34790a28 None None] MFA Rules not processed for user `fb2e6bc90b3a4f198ca5495a32e4faa9`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:27.120390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-65c81c5c-a878-4ac5-bd03-14ac34790a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:27.409553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 876/1745] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 435 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:27.492303 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f4d8a-2436-43fb-a1ff-9bc1befdba91 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:27.495175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b78f4d8a-2436-43fb-a1ff-9bc1befdba91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:27.666665 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78f4d8a-2436-43fb-a1ff-9bc1befdba91 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:27.678614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682eaf54-1799-45fb-9c0e-fcd965fc00d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:27.679449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682eaf54-1799-45fb-9c0e-fcd965fc00d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:27.679906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682eaf54-1799-45fb-9c0e-fcd965fc00d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:27.680351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682eaf54-1799-45fb-9c0e-fcd965fc00d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:27.691612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:27.863627 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682eaf54-1799-45fb-9c0e-fcd965fc00d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:27.866196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682eaf54-1799-45fb-9c0e-fcd965fc00d7 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:27.890148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 870/1746] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:27.973736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e129c82-8c35-4485-9b48-793f4dc182cc None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:27.974350 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e129c82-8c35-4485-9b48-793f4dc182cc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:27.974350 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e129c82-8c35-4485-9b48-793f4dc182cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:28.037036 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-1e129c82-8c35-4485-9b48-793f4dc182cc None None] MFA Rules not processed for user `093edc8c192c469f9532a45bb3326612`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:28.039124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1e129c82-8c35-4485-9b48-793f4dc182cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:28.043955 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682eaf54-1799-45fb-9c0e-fcd965fc00d7 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:28.045482 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-682eaf54-1799-45fb-9c0e-fcd965fc00d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:28.071701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682eaf54-1799-45fb-9c0e-fcd965fc00d7 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:28.097582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537ab22-4fbe-43f9-bd28-37b56b719360 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:28.098222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537ab22-4fbe-43f9-bd28-37b56b719360 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:28.098500 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537ab22-4fbe-43f9-bd28-37b56b719360 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:28.101112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537ab22-4fbe-43f9-bd28-37b56b719360 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:28.104152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d537ab22-4fbe-43f9-bd28-37b56b719360 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:28.245590 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 877/1747] 10.222.0.94 () {58 vars in 1015 bytes} [Wed Aug 31 19:12:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:28.306183 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d537ab22-4fbe-43f9-bd28-37b56b719360 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:28.308850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d537ab22-4fbe-43f9-bd28-37b56b719360 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:28.337063 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 871/1748] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:28 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:12:28.435656 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e129c82-8c35-4485-9b48-793f4dc182cc None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:28.437936 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1e129c82-8c35-4485-9b48-793f4dc182cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:28.468107 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e129c82-8c35-4485-9b48-793f4dc182cc None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:28.488895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aba610-00d1-429a-a820-3e09f302d97d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:28.490881 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aba610-00d1-429a-a820-3e09f302d97d service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:28.492608 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aba610-00d1-429a-a820-3e09f302d97d service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:28.494847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aba610-00d1-429a-a820-3e09f302d97d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:28.526791 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aba610-00d1-429a-a820-3e09f302d97d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:28.533737 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aba610-00d1-429a-a820-3e09f302d97d service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:28.574946 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 878/1749] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:30.161935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537ab22-4fbe-43f9-bd28-37b56b719360 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:30.164289 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d537ab22-4fbe-43f9-bd28-37b56b719360 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:30.457620 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537ab22-4fbe-43f9-bd28-37b56b719360 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:30.492141 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27733e05-8b84-438e-80c3-42213e7f5517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:30.496624 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27733e05-8b84-438e-80c3-42213e7f5517 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:30.497205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27733e05-8b84-438e-80c3-42213e7f5517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:30.499142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27733e05-8b84-438e-80c3-42213e7f5517 admin admin] PATH_INFO: `/v3/users/5ff8897a5def4518a310ce8441eb4efb` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:30.508410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27733e05-8b84-438e-80c3-42213e7f5517 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ff8897a5def4518a310ce8441eb4efb)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:30.519397 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27733e05-8b84-438e-80c3-42213e7f5517 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:30.665342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-27733e05-8b84-438e-80c3-42213e7f5517 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ff8897a5def4518a310ce8441eb4efb'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:30.667500 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:30.698304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-27733e05-8b84-438e-80c3-42213e7f5517 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ff8897a5def4518a310ce8441eb4efb'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:30.739016 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-27733e05-8b84-438e-80c3-42213e7f5517 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ff8897a5def4518a310ce8441eb4efb'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:30.739822 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-27733e05-8b84-438e-80c3-42213e7f5517 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ff8897a5def4518a310ce8441eb4efb'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:30.742058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-27733e05-8b84-438e-80c3-42213e7f5517 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ff8897a5def4518a310ce8441eb4efb'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:30.789759 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 872/1750] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:30 2022] DELETE /identity/v3/users/5ff8897a5def4518a310ce8441eb4efb => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:30.794810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5b7f9-a2f5-44c0-a3eb-29b76fceea94 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:30.795367 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5b7f9-a2f5-44c0-a3eb-29b76fceea94 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:30.795367 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5b7f9-a2f5-44c0-a3eb-29b76fceea94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:30.839079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d8a5b7f9-a2f5-44c0-a3eb-29b76fceea94 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:30.840667 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d8a5b7f9-a2f5-44c0-a3eb-29b76fceea94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:31.110029 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:31.138218 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 879/1751] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:31.174816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27733e05-8b84-438e-80c3-42213e7f5517 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:31.182418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-27733e05-8b84-438e-80c3-42213e7f5517 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:31.384975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27733e05-8b84-438e-80c3-42213e7f5517 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:31.390288 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5b7f9-a2f5-44c0-a3eb-29b76fceea94 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:31.391800 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d8a5b7f9-a2f5-44c0-a3eb-29b76fceea94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:31.411220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f6010-7914-4448-983b-058888a8b92e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:31.411862 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f6010-7914-4448-983b-058888a8b92e service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:31.412138 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f6010-7914-4448-983b-058888a8b92e service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:31.412404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f6010-7914-4448-983b-058888a8b92e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:31.424367 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5b7f9-a2f5-44c0-a3eb-29b76fceea94 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:31.443907 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d929cad-4f41-4df3-8572-105e54159cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:31.448515 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d929cad-4f41-4df3-8572-105e54159cc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:31.449724 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d929cad-4f41-4df3-8572-105e54159cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:31.451596 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d929cad-4f41-4df3-8572-105e54159cc0 admin admin] PATH_INFO: `/v3/users/c29df84822474a41942bcc8d761fbb34` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:31.455740 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d929cad-4f41-4df3-8572-105e54159cc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c29df84822474a41942bcc8d761fbb34)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:31.472684 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d929cad-4f41-4df3-8572-105e54159cc0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:31.474116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f6010-7914-4448-983b-058888a8b92e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:31.481212 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10f6010-7914-4448-983b-058888a8b92e service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:31.518928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 873/1752] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:12:31.665793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3d929cad-4f41-4df3-8572-105e54159cc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c29df84822474a41942bcc8d761fbb34'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:31.688127 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3d929cad-4f41-4df3-8572-105e54159cc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c29df84822474a41942bcc8d761fbb34'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:31.716712 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3d929cad-4f41-4df3-8572-105e54159cc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c29df84822474a41942bcc8d761fbb34'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:31.717346 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3d929cad-4f41-4df3-8572-105e54159cc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c29df84822474a41942bcc8d761fbb34'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:31.742878 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3d929cad-4f41-4df3-8572-105e54159cc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c29df84822474a41942bcc8d761fbb34'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:31.773513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 880/1753] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:31 2022] DELETE /identity/v3/users/c29df84822474a41942bcc8d761fbb34 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:12:31.775067 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c746ae-4dff-479a-b7ee-1e32b381bf1f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:31.775394 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c746ae-4dff-479a-b7ee-1e32b381bf1f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:31.775671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c746ae-4dff-479a-b7ee-1e32b381bf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:31.812112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-62c746ae-4dff-479a-b7ee-1e32b381bf1f None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:31.813865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-62c746ae-4dff-479a-b7ee-1e32b381bf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:31.966434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 874/1754] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:12:32.022006 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d929cad-4f41-4df3-8572-105e54159cc0 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:32.023573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3d929cad-4f41-4df3-8572-105e54159cc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:32.158784 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d929cad-4f41-4df3-8572-105e54159cc0 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:32.166817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022fe8b3-7d71-4b2f-896a-413ef4b25ed9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:32.167446 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fe8b3-7d71-4b2f-896a-413ef4b25ed9 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:32.168541 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fe8b3-7d71-4b2f-896a-413ef4b25ed9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:32.169065 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022fe8b3-7d71-4b2f-896a-413ef4b25ed9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:32.194957 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022fe8b3-7d71-4b2f-896a-413ef4b25ed9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:32.196430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022fe8b3-7d71-4b2f-896a-413ef4b25ed9 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:32.239457 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 881/1755] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:32.558549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c746ae-4dff-479a-b7ee-1e32b381bf1f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:32.562229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-62c746ae-4dff-479a-b7ee-1e32b381bf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:32.584090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c746ae-4dff-479a-b7ee-1e32b381bf1f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:32.590964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b03ff-b0af-439e-a058-85b4706ab91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:32.591730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b03ff-b0af-439e-a058-85b4706ab91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:32.592138 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b03ff-b0af-439e-a058-85b4706ab91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:32.592636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b03ff-b0af-439e-a058-85b4706ab91e admin admin] PATH_INFO: `/v3/projects/4c81000f401e4acfbb5264e1834f8fb7` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:32.594315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b03ff-b0af-439e-a058-85b4706ab91e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c81000f401e4acfbb5264e1834f8fb7)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:32.596632 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b03ff-b0af-439e-a058-85b4706ab91e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:32.693348 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-dd4b03ff-b0af-439e-a058-85b4706ab91e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c81000f401e4acfbb5264e1834f8fb7', 'request_id': 'req-dd4b03ff-b0af-439e-a058-85b4706ab91e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:32.711564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-dd4b03ff-b0af-439e-a058-85b4706ab91e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c81000f401e4acfbb5264e1834f8fb7', 'request_id': 'req-dd4b03ff-b0af-439e-a058-85b4706ab91e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:32.722210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:32.724299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-dd4b03ff-b0af-439e-a058-85b4706ab91e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c81000f401e4acfbb5264e1834f8fb7', 'request_id': 'req-dd4b03ff-b0af-439e-a058-85b4706ab91e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:32.767227 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-dd4b03ff-b0af-439e-a058-85b4706ab91e admin admin] The token cache is being invalidate because project 4c81000f401e4acfbb5264e1834f8fb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:32.767648 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-dd4b03ff-b0af-439e-a058-85b4706ab91e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:32.768314 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-dd4b03ff-b0af-439e-a058-85b4706ab91e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:32.774241 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022fe8b3-7d71-4b2f-896a-413ef4b25ed9 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:32.776056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 875/1756] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:32 2022] DELETE /identity/v3/projects/4c81000f401e4acfbb5264e1834f8fb7 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:32.786647 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022fe8b3-7d71-4b2f-896a-413ef4b25ed9 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:32.947785 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bc7f6-a39c-4428-ade1-565fa1bd6e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:32.955024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bc7f6-a39c-4428-ade1-565fa1bd6e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:32.955618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bc7f6-a39c-4428-ade1-565fa1bd6e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:32.956414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bc7f6-a39c-4428-ade1-565fa1bd6e6c admin admin] PATH_INFO: `/v3/users/2947bd5cf3654753914dfe9702295cb5` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:32.958325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bc7f6-a39c-4428-ade1-565fa1bd6e6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2947bd5cf3654753914dfe9702295cb5)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:32.960898 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bc7f6-a39c-4428-ade1-565fa1bd6e6c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:33.002399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:33.080127 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b03ff-b0af-439e-a058-85b4706ab91e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:33.082274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-dd4b03ff-b0af-439e-a058-85b4706ab91e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:33.086419 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-809bc7f6-a39c-4428-ade1-565fa1bd6e6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2947bd5cf3654753914dfe9702295cb5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.108989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b03ff-b0af-439e-a058-85b4706ab91e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:33.117594 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f9103a-9d12-42d3-989b-e4948c4d0261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:33.118640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9103a-9d12-42d3-989b-e4948c4d0261 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:33.119647 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9103a-9d12-42d3-989b-e4948c4d0261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:33.120034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-809bc7f6-a39c-4428-ade1-565fa1bd6e6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2947bd5cf3654753914dfe9702295cb5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.120304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f9103a-9d12-42d3-989b-e4948c4d0261 admin admin] PATH_INFO: `/v3/users/7dfd0bfbf2c0457c96725f0e737b0885` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:33.122946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9103a-9d12-42d3-989b-e4948c4d0261 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dfd0bfbf2c0457c96725f0e737b0885)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:33.126113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f9103a-9d12-42d3-989b-e4948c4d0261 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:33.137577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-809bc7f6-a39c-4428-ade1-565fa1bd6e6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2947bd5cf3654753914dfe9702295cb5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.142089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-809bc7f6-a39c-4428-ade1-565fa1bd6e6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2947bd5cf3654753914dfe9702295cb5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.166707 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-809bc7f6-a39c-4428-ade1-565fa1bd6e6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2947bd5cf3654753914dfe9702295cb5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.183812 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 882/1757] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:32 2022] DELETE /identity/v3/users/2947bd5cf3654753914dfe9702295cb5 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:12:33.184859 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bc7f6-a39c-4428-ade1-565fa1bd6e6c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:33.199768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-809bc7f6-a39c-4428-ade1-565fa1bd6e6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:33.215031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-46f9103a-9d12-42d3-989b-e4948c4d0261 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dfd0bfbf2c0457c96725f0e737b0885'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.224138 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-46f9103a-9d12-42d3-989b-e4948c4d0261 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dfd0bfbf2c0457c96725f0e737b0885'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.249608 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-46f9103a-9d12-42d3-989b-e4948c4d0261 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dfd0bfbf2c0457c96725f0e737b0885'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.250461 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-46f9103a-9d12-42d3-989b-e4948c4d0261 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dfd0bfbf2c0457c96725f0e737b0885'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.251187 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-46f9103a-9d12-42d3-989b-e4948c4d0261 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dfd0bfbf2c0457c96725f0e737b0885'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.284445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 876/1758] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:33 2022] DELETE /identity/v3/users/7dfd0bfbf2c0457c96725f0e737b0885 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:12:33.288572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd8632-63af-4ad3-8e44-8f1773012497 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:33.288952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd8632-63af-4ad3-8e44-8f1773012497 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:33.289266 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd8632-63af-4ad3-8e44-8f1773012497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:33.313482 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bc7f6-a39c-4428-ade1-565fa1bd6e6c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:33.316766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-57dd8632-63af-4ad3-8e44-8f1773012497 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:33.318679 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-57dd8632-63af-4ad3-8e44-8f1773012497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:33.395802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efe563a-eed1-4748-8293-551316ca8676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:33.396445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe563a-eed1-4748-8293-551316ca8676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:33.396722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe563a-eed1-4748-8293-551316ca8676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:33.396990 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efe563a-eed1-4748-8293-551316ca8676 admin admin] PATH_INFO: `/v3/projects/c6ef78ed6d034a1bb69cc3c71446ac8b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:33.398142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe563a-eed1-4748-8293-551316ca8676 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6ef78ed6d034a1bb69cc3c71446ac8b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:33.400030 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efe563a-eed1-4748-8293-551316ca8676 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:33.426292 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 877/1759] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:33.443268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd8632-63af-4ad3-8e44-8f1773012497 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:33.446012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-57dd8632-63af-4ad3-8e44-8f1773012497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:33.457776 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:33.461959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7efe563a-eed1-4748-8293-551316ca8676 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6ef78ed6d034a1bb69cc3c71446ac8b', 'request_id': 'req-7efe563a-eed1-4748-8293-551316ca8676'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.471504 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7efe563a-eed1-4748-8293-551316ca8676 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6ef78ed6d034a1bb69cc3c71446ac8b', 'request_id': 'req-7efe563a-eed1-4748-8293-551316ca8676'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.481612 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7efe563a-eed1-4748-8293-551316ca8676 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6ef78ed6d034a1bb69cc3c71446ac8b', 'request_id': 'req-7efe563a-eed1-4748-8293-551316ca8676'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.504434 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7efe563a-eed1-4748-8293-551316ca8676 admin admin] The token cache is being invalidate because project c6ef78ed6d034a1bb69cc3c71446ac8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:33.504993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7efe563a-eed1-4748-8293-551316ca8676 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.510600 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7efe563a-eed1-4748-8293-551316ca8676 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.512820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 883/1760] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:33 2022] DELETE /identity/v3/projects/c6ef78ed6d034a1bb69cc3c71446ac8b => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:33.514268 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77d9f1-70c3-4c2c-8045-8ee343cf3544 None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:33.514554 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77d9f1-70c3-4c2c-8045-8ee343cf3544 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:33.515025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc77d9f1-70c3-4c2c-8045-8ee343cf3544 None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:33.517149 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 884/1761] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 19:12:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:33.517908 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77d9f1-70c3-4c2c-8045-8ee343cf3544 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:33.523762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bc77d9f1-70c3-4c2c-8045-8ee343cf3544 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:33.573664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd8632-63af-4ad3-8e44-8f1773012497 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:33.598040 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:33.632480 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc77d9f1-70c3-4c2c-8045-8ee343cf3544 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:33.643646 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532dd0e-231d-4338-8fc7-74e96b82333d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:33.644976 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532dd0e-231d-4338-8fc7-74e96b82333d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:33.645446 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532dd0e-231d-4338-8fc7-74e96b82333d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:33.645860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f532dd0e-231d-4338-8fc7-74e96b82333d admin admin] PATH_INFO: `/v3/users/0dff4a0a1ba14906b7d9e9481b34ca4e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:33.648088 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532dd0e-231d-4338-8fc7-74e96b82333d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dff4a0a1ba14906b7d9e9481b34ca4e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:33.651999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f532dd0e-231d-4338-8fc7-74e96b82333d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:33.659362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c426f5-5139-45e0-a356-5395c44ef357 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:33.660122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c426f5-5139-45e0-a356-5395c44ef357 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:33.660662 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c426f5-5139-45e0-a356-5395c44ef357 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:33.661054 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c426f5-5139-45e0-a356-5395c44ef357 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:33.663962 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b4c426f5-5139-45e0-a356-5395c44ef357 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:33.680389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c426f5-5139-45e0-a356-5395c44ef357 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:33.682588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c426f5-5139-45e0-a356-5395c44ef357 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:33.697133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 878/1762] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:12:33.699310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcf719-cf2b-443c-873e-2705b8499364 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:33.699654 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcf719-cf2b-443c-873e-2705b8499364 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:33.700534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bcf719-cf2b-443c-873e-2705b8499364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:33.727150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-47bcf719-cf2b-443c-873e-2705b8499364 None None] MFA Rules not processed for user `673a72ac9a9847e3812007b66ac7a5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:33.729619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-47bcf719-cf2b-443c-873e-2705b8499364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:33.731303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f532dd0e-231d-4338-8fc7-74e96b82333d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dff4a0a1ba14906b7d9e9481b34ca4e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.742952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f532dd0e-231d-4338-8fc7-74e96b82333d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dff4a0a1ba14906b7d9e9481b34ca4e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.754925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f532dd0e-231d-4338-8fc7-74e96b82333d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dff4a0a1ba14906b7d9e9481b34ca4e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.756003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f532dd0e-231d-4338-8fc7-74e96b82333d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dff4a0a1ba14906b7d9e9481b34ca4e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.775117 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f532dd0e-231d-4338-8fc7-74e96b82333d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dff4a0a1ba14906b7d9e9481b34ca4e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:33.795101 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 885/1763] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:33 2022] DELETE /identity/v3/users/0dff4a0a1ba14906b7d9e9481b34ca4e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:33.796815 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532dd0e-231d-4338-8fc7-74e96b82333d admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:33.798303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f532dd0e-231d-4338-8fc7-74e96b82333d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:33.893524 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 879/1764] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 19:12:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:33.925685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bcf719-cf2b-443c-873e-2705b8499364 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:33.928362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-47bcf719-cf2b-443c-873e-2705b8499364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:33.931584 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:33.950048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bcf719-cf2b-443c-873e-2705b8499364 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:33.961418 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f532dd0e-231d-4338-8fc7-74e96b82333d admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:33.969532 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7b8f6-d28d-4b8d-93f7-785e7ab1ef43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:33.971599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7b8f6-d28d-4b8d-93f7-785e7ab1ef43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:33.972268 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7b8f6-d28d-4b8d-93f7-785e7ab1ef43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:33.972898 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7b8f6-d28d-4b8d-93f7-785e7ab1ef43 admin admin] PATH_INFO: `/v3/projects/f8f4c294af4641a88e71e093731cad59` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:33.975023 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7b8f6-d28d-4b8d-93f7-785e7ab1ef43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8f4c294af4641a88e71e093731cad59)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:33.977418 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7b8f6-d28d-4b8d-93f7-785e7ab1ef43 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:34.064944 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f876f5-31e3-4d42-8727-0adcca4d702b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:34.066275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f876f5-31e3-4d42-8727-0adcca4d702b service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:34.068812 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c8d7b8f6-d28d-4b8d-93f7-785e7ab1ef43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8f4c294af4641a88e71e093731cad59', 'request_id': 'req-c8d7b8f6-d28d-4b8d-93f7-785e7ab1ef43'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.069777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f876f5-31e3-4d42-8727-0adcca4d702b service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:34.070160 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f876f5-31e3-4d42-8727-0adcca4d702b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:34.082213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f876f5-31e3-4d42-8727-0adcca4d702b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:34.089148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f876f5-31e3-4d42-8727-0adcca4d702b service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:34.093947 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c8d7b8f6-d28d-4b8d-93f7-785e7ab1ef43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8f4c294af4641a88e71e093731cad59', 'request_id': 'req-c8d7b8f6-d28d-4b8d-93f7-785e7ab1ef43'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.112718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c8d7b8f6-d28d-4b8d-93f7-785e7ab1ef43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8f4c294af4641a88e71e093731cad59', 'request_id': 'req-c8d7b8f6-d28d-4b8d-93f7-785e7ab1ef43'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.113110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 880/1765] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:34.127284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f876f5-31e3-4d42-8727-0adcca4d702b service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:34.128717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-01f876f5-31e3-4d42-8727-0adcca4d702b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:34.133779 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c8d7b8f6-d28d-4b8d-93f7-785e7ab1ef43 admin admin] The token cache is being invalidate because project f8f4c294af4641a88e71e093731cad59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:34.134078 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c8d7b8f6-d28d-4b8d-93f7-785e7ab1ef43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.135344 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c8d7b8f6-d28d-4b8d-93f7-785e7ab1ef43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.140407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 886/1766] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:33 2022] DELETE /identity/v3/projects/f8f4c294af4641a88e71e093731cad59 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:34.141783 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530d13c-cfcb-4050-b2a2-3dd930b74533 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:34.142072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530d13c-cfcb-4050-b2a2-3dd930b74533 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:34.142347 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3530d13c-cfcb-4050-b2a2-3dd930b74533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:34.153917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f876f5-31e3-4d42-8727-0adcca4d702b service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:34.173577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-3530d13c-cfcb-4050-b2a2-3dd930b74533 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:34.174878 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3530d13c-cfcb-4050-b2a2-3dd930b74533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:34.221774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:34.268481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9bf44-5538-4a00-9035-a27f44d22303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:34.269634 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9bf44-5538-4a00-9035-a27f44d22303 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:34.270376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9bf44-5538-4a00-9035-a27f44d22303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:34.271055 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9bf44-5538-4a00-9035-a27f44d22303 admin admin] PATH_INFO: `/v3/projects/a4dece16c7044a0883d30c1234adda44` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:34.272274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9bf44-5538-4a00-9035-a27f44d22303 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4dece16c7044a0883d30c1234adda44)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:34.274882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9bf44-5538-4a00-9035-a27f44d22303 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:34.311044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 887/1767] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:34.312940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530d13c-cfcb-4050-b2a2-3dd930b74533 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:34.313927 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3530d13c-cfcb-4050-b2a2-3dd930b74533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:34.334840 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3530d13c-cfcb-4050-b2a2-3dd930b74533 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:34.343163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379926e-6409-4552-adf1-444ac3ea3667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:34.343870 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379926e-6409-4552-adf1-444ac3ea3667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:34.344156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379926e-6409-4552-adf1-444ac3ea3667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:34.344458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379926e-6409-4552-adf1-444ac3ea3667 admin admin] PATH_INFO: `/v3/projects/e1087d1c9a9f4628bd7c3ea53958a3c9` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:34.345611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379926e-6409-4552-adf1-444ac3ea3667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1087d1c9a9f4628bd7c3ea53958a3c9)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:34.347528 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379926e-6409-4552-adf1-444ac3ea3667 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:34.371101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-1df9bf44-5538-4a00-9035-a27f44d22303 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4dece16c7044a0883d30c1234adda44', 'request_id': 'req-1df9bf44-5538-4a00-9035-a27f44d22303'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.385081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-1df9bf44-5538-4a00-9035-a27f44d22303 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4dece16c7044a0883d30c1234adda44', 'request_id': 'req-1df9bf44-5538-4a00-9035-a27f44d22303'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.396044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-1df9bf44-5538-4a00-9035-a27f44d22303 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4dece16c7044a0883d30c1234adda44', 'request_id': 'req-1df9bf44-5538-4a00-9035-a27f44d22303'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.409559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-1df9bf44-5538-4a00-9035-a27f44d22303 admin admin] The token cache is being invalidate because project a4dece16c7044a0883d30c1234adda44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:34.409828 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-1df9bf44-5538-4a00-9035-a27f44d22303 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.411394 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-1df9bf44-5538-4a00-9035-a27f44d22303 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.413997 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 881/1768] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:34 2022] DELETE /identity/v3/projects/a4dece16c7044a0883d30c1234adda44 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:34.415339 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1309ad-e583-405a-8228-ebddf44834e8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:34.416277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1309ad-e583-405a-8228-ebddf44834e8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:34.416551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1309ad-e583-405a-8228-ebddf44834e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:34.419760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7379926e-6409-4552-adf1-444ac3ea3667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1087d1c9a9f4628bd7c3ea53958a3c9', 'request_id': 'req-7379926e-6409-4552-adf1-444ac3ea3667'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.426752 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7379926e-6409-4552-adf1-444ac3ea3667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1087d1c9a9f4628bd7c3ea53958a3c9', 'request_id': 'req-7379926e-6409-4552-adf1-444ac3ea3667'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.436265 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7379926e-6409-4552-adf1-444ac3ea3667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1087d1c9a9f4628bd7c3ea53958a3c9', 'request_id': 'req-7379926e-6409-4552-adf1-444ac3ea3667'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.440873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-fe1309ad-e583-405a-8228-ebddf44834e8 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:34.442272 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-fe1309ad-e583-405a-8228-ebddf44834e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:34.450117 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7379926e-6409-4552-adf1-444ac3ea3667 admin admin] The token cache is being invalidate because project e1087d1c9a9f4628bd7c3ea53958a3c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:34.450464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7379926e-6409-4552-adf1-444ac3ea3667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.451109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7379926e-6409-4552-adf1-444ac3ea3667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.452674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 888/1769] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:34 2022] DELETE /identity/v3/projects/e1087d1c9a9f4628bd7c3ea53958a3c9 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:34.456013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379926e-6409-4552-adf1-444ac3ea3667 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:34.456013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7379926e-6409-4552-adf1-444ac3ea3667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:34.463755 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:34.517727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 882/1770] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:34.519091 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae9bf9-b5ce-49c5-8dc9-9ea022efdfb2 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:34.519440 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae9bf9-b5ce-49c5-8dc9-9ea022efdfb2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:34.519785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae9bf9-b5ce-49c5-8dc9-9ea022efdfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:34.544332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-90ae9bf9-b5ce-49c5-8dc9-9ea022efdfb2 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:34.544935 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379926e-6409-4552-adf1-444ac3ea3667 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:34.545920 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-90ae9bf9-b5ce-49c5-8dc9-9ea022efdfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:34.551503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b5c42-ca97-4a76-ba81-0627ce18e25d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:34.552153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b5c42-ca97-4a76-ba81-0627ce18e25d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:34.552431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b5c42-ca97-4a76-ba81-0627ce18e25d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:34.552709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b5c42-ca97-4a76-ba81-0627ce18e25d admin admin] PATH_INFO: `/v3/users/b302a95cd1404b1b80d534db57e78b3e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:34.553901 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b5c42-ca97-4a76-ba81-0627ce18e25d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b302a95cd1404b1b80d534db57e78b3e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:34.555876 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9b5c42-ca97-4a76-ba81-0627ce18e25d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:34.562779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 883/1771] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:34.562779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae9bf9-b5ce-49c5-8dc9-9ea022efdfb2 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:34.573753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae9bf9-b5ce-49c5-8dc9-9ea022efdfb2 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:34.577979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e3911-9aed-4305-bdcb-95908e93c0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:34.578579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e3911-9aed-4305-bdcb-95908e93c0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:34.578857 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e3911-9aed-4305-bdcb-95908e93c0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:34.579132 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e3911-9aed-4305-bdcb-95908e93c0a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:34.579776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e3911-9aed-4305-bdcb-95908e93c0a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:34.580072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e3911-9aed-4305-bdcb-95908e93c0a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:34.581705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886e3911-9aed-4305-bdcb-95908e93c0a4 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:34.596407 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 884/1772] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:12:34.598537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a210a0-83e5-4311-94ec-8dc377b8d14b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:34.598537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a210a0-83e5-4311-94ec-8dc377b8d14b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:34.598706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a210a0-83e5-4311-94ec-8dc377b8d14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:34.615951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ea9b5c42-ca97-4a76-ba81-0627ce18e25d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b302a95cd1404b1b80d534db57e78b3e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.622078 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b4a210a0-83e5-4311-94ec-8dc377b8d14b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:34.623285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b4a210a0-83e5-4311-94ec-8dc377b8d14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:34.626959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ea9b5c42-ca97-4a76-ba81-0627ce18e25d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b302a95cd1404b1b80d534db57e78b3e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.631537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 885/1773] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:34.635862 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a210a0-83e5-4311-94ec-8dc377b8d14b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:34.636240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ea9b5c42-ca97-4a76-ba81-0627ce18e25d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b302a95cd1404b1b80d534db57e78b3e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.636882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ea9b5c42-ca97-4a76-ba81-0627ce18e25d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b302a95cd1404b1b80d534db57e78b3e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.647077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a210a0-83e5-4311-94ec-8dc377b8d14b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:34.649823 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ea9b5c42-ca97-4a76-ba81-0627ce18e25d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b302a95cd1404b1b80d534db57e78b3e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:34.653281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f66f27-614a-4c58-a98f-5371b3207344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:34.653894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f66f27-614a-4c58-a98f-5371b3207344 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:34.654174 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f66f27-614a-4c58-a98f-5371b3207344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:34.654442 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f66f27-614a-4c58-a98f-5371b3207344 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:34.655066 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f66f27-614a-4c58-a98f-5371b3207344 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:34.659266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 889/1774] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:34 2022] DELETE /identity/v3/users/b302a95cd1404b1b80d534db57e78b3e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:34.659395 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f66f27-614a-4c58-a98f-5371b3207344 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:34.659481 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b5c42-ca97-4a76-ba81-0627ce18e25d admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:34.662272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ea9b5c42-ca97-4a76-ba81-0627ce18e25d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:34.697908 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 886/1775] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:34 2022] POST /identity/v3/projects => generated 387 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:34.702094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f66f27-614a-4c58-a98f-5371b3207344 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:34.704152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-90f66f27-614a-4c58-a98f-5371b3207344 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:34.744562 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b5c42-ca97-4a76-ba81-0627ce18e25d admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:34.756710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:34.807683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:34.808204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f66f27-614a-4c58-a98f-5371b3207344 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:34.816210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c07085-44ef-45b5-b7f7-4837fad3ad66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:34.816865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c07085-44ef-45b5-b7f7-4837fad3ad66 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:34.817313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c07085-44ef-45b5-b7f7-4837fad3ad66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:34.817800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c07085-44ef-45b5-b7f7-4837fad3ad66 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:34.818448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c07085-44ef-45b5-b7f7-4837fad3ad66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:34.820451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c07085-44ef-45b5-b7f7-4837fad3ad66 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:34.844198 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290abe78-dc2e-4522-a261-66c9fed295a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:34.844827 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290abe78-dc2e-4522-a261-66c9fed295a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:34.845157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290abe78-dc2e-4522-a261-66c9fed295a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:34.845461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290abe78-dc2e-4522-a261-66c9fed295a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:34.848760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 887/1776] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:34 2022] POST /identity/v3/users => generated 294 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:12:34.848890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290abe78-dc2e-4522-a261-66c9fed295a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:34.848890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290abe78-dc2e-4522-a261-66c9fed295a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:34.848890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290abe78-dc2e-4522-a261-66c9fed295a0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:34.849946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c07085-44ef-45b5-b7f7-4837fad3ad66 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:34.868502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c07085-44ef-45b5-b7f7-4837fad3ad66 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:34.877811 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73901a86-c79b-43b0-bb26-d8811853ff48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:34.878426 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73901a86-c79b-43b0-bb26-d8811853ff48 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:34.878702 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73901a86-c79b-43b0-bb26-d8811853ff48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:34.881238 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 890/1777] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:12:34.881380 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73901a86-c79b-43b0-bb26-d8811853ff48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:34.881380 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73901a86-c79b-43b0-bb26-d8811853ff48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:34.882615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73901a86-c79b-43b0-bb26-d8811853ff48 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:34.884776 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290abe78-dc2e-4522-a261-66c9fed295a0 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:34.900173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 888/1778] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:34.902536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290abe78-dc2e-4522-a261-66c9fed295a0 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:34.909223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73901a86-c79b-43b0-bb26-d8811853ff48 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:34.909790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1420ac-5e12-4d1a-a863-b67694d12a3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:34.910903 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1420ac-5e12-4d1a-a863-b67694d12a3a admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:34.911433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1420ac-5e12-4d1a-a863-b67694d12a3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:34.911890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1420ac-5e12-4d1a-a863-b67694d12a3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:34.912703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1420ac-5e12-4d1a-a863-b67694d12a3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:34.916405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1420ac-5e12-4d1a-a863-b67694d12a3a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:34.923015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73901a86-c79b-43b0-bb26-d8811853ff48 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:34.938416 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98165118-2f20-48f6-8adc-d2d9d5d72e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:34.939606 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98165118-2f20-48f6-8adc-d2d9d5d72e00 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:34.940510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98165118-2f20-48f6-8adc-d2d9d5d72e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:34.940684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98165118-2f20-48f6-8adc-d2d9d5d72e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:34.945517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98165118-2f20-48f6-8adc-d2d9d5d72e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:34.950623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98165118-2f20-48f6-8adc-d2d9d5d72e00 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:34.968619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 891/1779] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:34 2022] POST /identity/v3/projects => generated 387 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:12:34.972783 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1420ac-5e12-4d1a-a863-b67694d12a3a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:34.975028 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 889/1780] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:34 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:34.987314 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98165118-2f20-48f6-8adc-d2d9d5d72e00 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:35.004801 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98165118-2f20-48f6-8adc-d2d9d5d72e00 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:35.133556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1420ac-5e12-4d1a-a863-b67694d12a3a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:35.138015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e895a0-686a-437d-b008-718f323a83be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:35.138954 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e895a0-686a-437d-b008-718f323a83be admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:35.139284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e895a0-686a-437d-b008-718f323a83be admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:35.139566 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e895a0-686a-437d-b008-718f323a83be admin admin] PATH_INFO: `/v3/projects/6b1888d89fb2437fa3f4dc88d696ebfe/users/d9f00d48e9734b67be5e9b474bf40461/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:35.140263 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78d64c-3363-44ec-92c5-768fe6b38f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:35.140966 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78d64c-3363-44ec-92c5-768fe6b38f3b admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:35.141288 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78d64c-3363-44ec-92c5-768fe6b38f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:35.141559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78d64c-3363-44ec-92c5-768fe6b38f3b admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:35.142191 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78d64c-3363-44ec-92c5-768fe6b38f3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:35.144191 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c78d64c-3363-44ec-92c5-768fe6b38f3b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:35.174106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 892/1781] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:34 2022] POST /identity/v3/users => generated 294 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:35.174984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78d64c-3363-44ec-92c5-768fe6b38f3b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:35.177853 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4c78d64c-3363-44ec-92c5-768fe6b38f3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:35.188754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e895a0-686a-437d-b008-718f323a83be admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b1888d89fb2437fa3f4dc88d696ebfe, user_id=d9f00d48e9734b67be5e9b474bf40461, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:35.191637 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e895a0-686a-437d-b008-718f323a83be admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:35.199929 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78d64c-3363-44ec-92c5-768fe6b38f3b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:35.220290 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b784ba3-fbf8-4a45-991e-9e84833af3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:35.220969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b784ba3-fbf8-4a45-991e-9e84833af3a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:35.222193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b784ba3-fbf8-4a45-991e-9e84833af3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:35.222477 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b784ba3-fbf8-4a45-991e-9e84833af3a1 admin admin] PATH_INFO: `/v3/projects/e7a629d72e234343aaf34e8627bf8124` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:35.224110 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b784ba3-fbf8-4a45-991e-9e84833af3a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7a629d72e234343aaf34e8627bf8124)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:35.227060 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b784ba3-fbf8-4a45-991e-9e84833af3a1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:35.230975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 890/1782] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:34 2022] PUT /identity/v3/projects/6b1888d89fb2437fa3f4dc88d696ebfe/users/d9f00d48e9734b67be5e9b474bf40461/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:35.234145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e895a0-686a-437d-b008-718f323a83be admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:35.321351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e895a0-686a-437d-b008-718f323a83be admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:35.342563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386b8e55-c891-403f-b1f6-bc50e6a0befc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:35.346862 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b8e55-c891-403f-b1f6-bc50e6a0befc admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:35.347166 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b8e55-c891-403f-b1f6-bc50e6a0befc admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:35.348253 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386b8e55-c891-403f-b1f6-bc50e6a0befc admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:35.348996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b8e55-c891-403f-b1f6-bc50e6a0befc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:35.350646 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386b8e55-c891-403f-b1f6-bc50e6a0befc admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:35.362786 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6b784ba3-fbf8-4a45-991e-9e84833af3a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7a629d72e234343aaf34e8627bf8124', 'request_id': 'req-6b784ba3-fbf8-4a45-991e-9e84833af3a1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:35.369694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6b784ba3-fbf8-4a45-991e-9e84833af3a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7a629d72e234343aaf34e8627bf8124', 'request_id': 'req-6b784ba3-fbf8-4a45-991e-9e84833af3a1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:35.376359 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6b784ba3-fbf8-4a45-991e-9e84833af3a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7a629d72e234343aaf34e8627bf8124', 'request_id': 'req-6b784ba3-fbf8-4a45-991e-9e84833af3a1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:35.379004 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 891/1783] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:35.382784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc172a20-32a7-4958-a567-fce6292db67f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:35.383259 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc172a20-32a7-4958-a567-fce6292db67f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:35.383737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc172a20-32a7-4958-a567-fce6292db67f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:35.398275 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6b784ba3-fbf8-4a45-991e-9e84833af3a1 admin admin] The token cache is being invalidate because project e7a629d72e234343aaf34e8627bf8124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:35.398638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6b784ba3-fbf8-4a45-991e-9e84833af3a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:35.399342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6b784ba3-fbf8-4a45-991e-9e84833af3a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:35.401739 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 893/1784] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:35 2022] DELETE /identity/v3/projects/e7a629d72e234343aaf34e8627bf8124 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:35.402641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b784ba3-fbf8-4a45-991e-9e84833af3a1 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:35.404113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6b784ba3-fbf8-4a45-991e-9e84833af3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:35.439227 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:35.450858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-bc172a20-32a7-4958-a567-fce6292db67f None None] MFA Rules not processed for user `d9f00d48e9734b67be5e9b474bf40461`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:35.452593 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:35.453081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bc172a20-32a7-4958-a567-fce6292db67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:35.556890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b784ba3-fbf8-4a45-991e-9e84833af3a1 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:35.583819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddca6b7-79dc-42b1-9ebf-b8d7f23ea1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:35.584474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddca6b7-79dc-42b1-9ebf-b8d7f23ea1a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:35.584759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddca6b7-79dc-42b1-9ebf-b8d7f23ea1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:35.585034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddca6b7-79dc-42b1-9ebf-b8d7f23ea1a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:35.585725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddca6b7-79dc-42b1-9ebf-b8d7f23ea1a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:35.589880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddca6b7-79dc-42b1-9ebf-b8d7f23ea1a3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:35.611140 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 892/1785] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:35.613447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc172a20-32a7-4958-a567-fce6292db67f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:35.618733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 894/1786] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:35.618733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddca6b7-79dc-42b1-9ebf-b8d7f23ea1a3 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:35.619601 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bc172a20-32a7-4958-a567-fce6292db67f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:35.620002 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bddca6b7-79dc-42b1-9ebf-b8d7f23ea1a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:35.637440 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc172a20-32a7-4958-a567-fce6292db67f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:35.643431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3c038-5403-49a6-b310-0c2f137435e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:35.643599 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3c038-5403-49a6-b310-0c2f137435e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:35.643771 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3c038-5403-49a6-b310-0c2f137435e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:35.644049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf3c038-5403-49a6-b310-0c2f137435e7 admin admin] PATH_INFO: `/v3/users/35978b8fcccc4b57bb8d4a15032e708a` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:35.645335 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3c038-5403-49a6-b310-0c2f137435e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35978b8fcccc4b57bb8d4a15032e708a)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:35.647349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf3c038-5403-49a6-b310-0c2f137435e7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:35.655399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddca6b7-79dc-42b1-9ebf-b8d7f23ea1a3 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:35.661040 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118324f3-4f14-4209-8ce2-c64f7d82ab2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:35.661690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118324f3-4f14-4209-8ce2-c64f7d82ab2f admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:35.661973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118324f3-4f14-4209-8ce2-c64f7d82ab2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:35.662478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118324f3-4f14-4209-8ce2-c64f7d82ab2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:35.662879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118324f3-4f14-4209-8ce2-c64f7d82ab2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:35.664780 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118324f3-4f14-4209-8ce2-c64f7d82ab2f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:35.716762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 895/1787] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:35 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:35.718091 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118324f3-4f14-4209-8ce2-c64f7d82ab2f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:35.725651 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5cf3c038-5403-49a6-b310-0c2f137435e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35978b8fcccc4b57bb8d4a15032e708a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:35.736162 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5cf3c038-5403-49a6-b310-0c2f137435e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35978b8fcccc4b57bb8d4a15032e708a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:35.752347 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5cf3c038-5403-49a6-b310-0c2f137435e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35978b8fcccc4b57bb8d4a15032e708a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:35.753456 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5cf3c038-5403-49a6-b310-0c2f137435e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35978b8fcccc4b57bb8d4a15032e708a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:35.754223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5cf3c038-5403-49a6-b310-0c2f137435e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35978b8fcccc4b57bb8d4a15032e708a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:35.769769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 893/1788] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:35 2022] DELETE /identity/v3/users/35978b8fcccc4b57bb8d4a15032e708a => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:35.770567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3c038-5403-49a6-b310-0c2f137435e7 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:35.772020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5cf3c038-5403-49a6-b310-0c2f137435e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:35.801697 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118324f3-4f14-4209-8ce2-c64f7d82ab2f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:35.865035 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf3c038-5403-49a6-b310-0c2f137435e7 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:35.874073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f471ac-d046-4c22-8d05-a2254671b3bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:35.875912 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f471ac-d046-4c22-8d05-a2254671b3bd admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:35.876238 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f471ac-d046-4c22-8d05-a2254671b3bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:35.876513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f471ac-d046-4c22-8d05-a2254671b3bd admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:35.878267 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f471ac-d046-4c22-8d05-a2254671b3bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:35.883548 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3886f-6281-46d1-ad97-659835d9c604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:35.884228 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3886f-6281-46d1-ad97-659835d9c604 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:35.885042 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3886f-6281-46d1-ad97-659835d9c604 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:35.885896 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3886f-6281-46d1-ad97-659835d9c604 admin admin] PATH_INFO: `/v3/projects/81bcc34e27e8473da924e2bed9783eb9/users/cd659521cae04153ae616776358bba66/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:35.886155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f471ac-d046-4c22-8d05-a2254671b3bd admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:35.907798 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3886f-6281-46d1-ad97-659835d9c604 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81bcc34e27e8473da924e2bed9783eb9, user_id=cd659521cae04153ae616776358bba66, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:35.910860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3886f-6281-46d1-ad97-659835d9c604 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:35.921870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:35.932371 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 894/1789] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:35 2022] POST /identity/v3/users => generated 300 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:35.940061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f471ac-d046-4c22-8d05-a2254671b3bd admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:35.961119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 896/1790] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:35 2022] PUT /identity/v3/projects/81bcc34e27e8473da924e2bed9783eb9/users/cd659521cae04153ae616776358bba66/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:35.961119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bed872-bc4f-43e7-b2b4-da1a8510d6f6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:35.961119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bed872-bc4f-43e7-b2b4-da1a8510d6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:35.961119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bed872-bc4f-43e7-b2b4-da1a8510d6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:35.963324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f471ac-d046-4c22-8d05-a2254671b3bd admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:36.004255 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-17bed872-bc4f-43e7-b2b4-da1a8510d6f6 None None] MFA Rules not processed for user `cd659521cae04153ae616776358bba66`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:36.006435 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-17bed872-bc4f-43e7-b2b4-da1a8510d6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:36.019867 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:36.058668 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9fbe0-9e29-4fbd-a103-44d952cabf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:36.059962 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9fbe0-9e29-4fbd-a103-44d952cabf16 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.060203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9fbe0-9e29-4fbd-a103-44d952cabf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.060578 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b9fbe0-9e29-4fbd-a103-44d952cabf16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:36.061365 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9fbe0-9e29-4fbd-a103-44d952cabf16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:36.063815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b9fbe0-9e29-4fbd-a103-44d952cabf16 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:36.085132 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 895/1791] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:36.086336 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9fbe0-9e29-4fbd-a103-44d952cabf16 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:36.099743 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b9fbe0-9e29-4fbd-a103-44d952cabf16 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:36.109952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465c424-0bcf-4d66-9b2f-ef67a7ca6491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:36.109952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465c424-0bcf-4d66-9b2f-ef67a7ca6491 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.109952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465c424-0bcf-4d66-9b2f-ef67a7ca6491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.109952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6465c424-0bcf-4d66-9b2f-ef67a7ca6491 admin admin] PATH_INFO: `/v3/projects/0d22285f09b94145bab87e8727008288/users/b90f8980596849debe472615e7030874/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:36.116835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 897/1792] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:12:36.121831 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bed872-bc4f-43e7-b2b4-da1a8510d6f6 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:36.123156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-17bed872-bc4f-43e7-b2b4-da1a8510d6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:36.130148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465c424-0bcf-4d66-9b2f-ef67a7ca6491 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d22285f09b94145bab87e8727008288, user_id=b90f8980596849debe472615e7030874, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:36.133273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6465c424-0bcf-4d66-9b2f-ef67a7ca6491 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:36.141144 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bed872-bc4f-43e7-b2b4-da1a8510d6f6 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:36.152697 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dca0ae-bb2b-47f5-94f4-84b1db50e4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:36.153075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 896/1793] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:36 2022] PUT /identity/v3/projects/0d22285f09b94145bab87e8727008288/users/b90f8980596849debe472615e7030874/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:36.156809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dca0ae-bb2b-47f5-94f4-84b1db50e4f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.156971 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465c424-0bcf-4d66-9b2f-ef67a7ca6491 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:36.157239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dca0ae-bb2b-47f5-94f4-84b1db50e4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.157734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dca0ae-bb2b-47f5-94f4-84b1db50e4f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:36.158724 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dca0ae-bb2b-47f5-94f4-84b1db50e4f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:36.160885 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91dca0ae-bb2b-47f5-94f4-84b1db50e4f3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:36.199886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 898/1794] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:36 2022] POST /identity/v3/projects => generated 387 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:12:36.204720 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dca0ae-bb2b-47f5-94f4-84b1db50e4f3 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:36.271678 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6465c424-0bcf-4d66-9b2f-ef67a7ca6491 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:36.318143 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91dca0ae-bb2b-47f5-94f4-84b1db50e4f3 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:36.323369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f99190-3eb2-4549-aa9f-c09ba5771972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:36.324042 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f99190-3eb2-4549-aa9f-c09ba5771972 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.324324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f99190-3eb2-4549-aa9f-c09ba5771972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.324594 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f99190-3eb2-4549-aa9f-c09ba5771972 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:36.325357 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f99190-3eb2-4549-aa9f-c09ba5771972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:36.327727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f99190-3eb2-4549-aa9f-c09ba5771972 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:36.362130 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 899/1795] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:36 2022] POST /identity/v3/users => generated 300 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:36.362130 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f99190-3eb2-4549-aa9f-c09ba5771972 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:36.362130 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-94f99190-3eb2-4549-aa9f-c09ba5771972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:36.371128 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3cecb-e062-4716-b3f8-5f1f81237625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:36.371980 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3cecb-e062-4716-b3f8-5f1f81237625 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.372459 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3cecb-e062-4716-b3f8-5f1f81237625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.372919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3cecb-e062-4716-b3f8-5f1f81237625 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:36.373779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3cecb-e062-4716-b3f8-5f1f81237625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:36.375529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3cecb-e062-4716-b3f8-5f1f81237625 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:36.383077 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f99190-3eb2-4549-aa9f-c09ba5771972 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:36.388662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:36.389453 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.389735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.390010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8 admin admin] PATH_INFO: `/v3/projects/e46e5e1cbd984396b231d616f6223617` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:36.391173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e46e5e1cbd984396b231d616f6223617)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:36.393234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 897/1796] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:36.393376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:36.393959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3cecb-e062-4716-b3f8-5f1f81237625 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:36.408012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3cecb-e062-4716-b3f8-5f1f81237625 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:36.413060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289d86f-4b65-460d-a972-a38f7621078c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:36.413697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289d86f-4b65-460d-a972-a38f7621078c admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.413975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289d86f-4b65-460d-a972-a38f7621078c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.414251 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1289d86f-4b65-460d-a972-a38f7621078c admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:36.414889 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289d86f-4b65-460d-a972-a38f7621078c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:36.416555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1289d86f-4b65-460d-a972-a38f7621078c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:36.428596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 898/1797] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:36.428596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289d86f-4b65-460d-a972-a38f7621078c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:36.445158 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:36.450420 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1289d86f-4b65-460d-a972-a38f7621078c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:36.463334 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e46e5e1cbd984396b231d616f6223617', 'request_id': 'req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:36.470255 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e46e5e1cbd984396b231d616f6223617', 'request_id': 'req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:36.476600 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e46e5e1cbd984396b231d616f6223617', 'request_id': 'req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:36.487981 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:36.492737 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8 admin admin] The token cache is being invalidate because project e46e5e1cbd984396b231d616f6223617 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:36.493215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:36.494338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:36.496558 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 900/1798] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:36 2022] DELETE /identity/v3/projects/e46e5e1cbd984396b231d616f6223617 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:36.497493 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:36.499428 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:36.530454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6539cb-3fa9-46f1-a1a3-0d958dcb695f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:36.531217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6539cb-3fa9-46f1-a1a3-0d958dcb695f admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.531616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6539cb-3fa9-46f1-a1a3-0d958dcb695f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.531963 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6539cb-3fa9-46f1-a1a3-0d958dcb695f admin admin] PATH_INFO: `/v3/projects/0d22285f09b94145bab87e8727008288/users/b90f8980596849debe472615e7030874/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:36.534059 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6539cb-3fa9-46f1-a1a3-0d958dcb695f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d22285f09b94145bab87e8727008288, user_id=b90f8980596849debe472615e7030874, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:36.536820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6539cb-3fa9-46f1-a1a3-0d958dcb695f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:36.549802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 899/1799] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:36 2022] PUT /identity/v3/projects/0d22285f09b94145bab87e8727008288/users/b90f8980596849debe472615e7030874/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:36.551167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92e6d0-58a5-4466-9733-3f7c89f21835 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.551450 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92e6d0-58a5-4466-9733-3f7c89f21835 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.551723 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a92e6d0-58a5-4466-9733-3f7c89f21835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:36.576235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-8a92e6d0-58a5-4466-9733-3f7c89f21835 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:36.577768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8a92e6d0-58a5-4466-9733-3f7c89f21835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:36.591539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cbf42b-2ade-4022-ab6e-e7e7073f7bf8 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:36.644372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 900/1800] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:36.645692 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1eb1f-fc69-45cf-9cc0-e77929b53369 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.645966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1eb1f-fc69-45cf-9cc0-e77929b53369 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.646236 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a1eb1f-fc69-45cf-9cc0-e77929b53369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:36.655233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83ab25-3974-4323-ab87-11e7e62d3fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:36.655899 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83ab25-3974-4323-ab87-11e7e62d3fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.656180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83ab25-3974-4323-ab87-11e7e62d3fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.656455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83ab25-3974-4323-ab87-11e7e62d3fb2 admin admin] PATH_INFO: `/v3/projects/d7206977944f4875a92b6ac6db3f1b58/users/905dff5f830546289a6dcb1d12004c92/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:36.671745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83ab25-3974-4323-ab87-11e7e62d3fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7206977944f4875a92b6ac6db3f1b58, user_id=905dff5f830546289a6dcb1d12004c92, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:36.674496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83ab25-3974-4323-ab87-11e7e62d3fb2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:36.677608 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e2a1eb1f-fc69-45cf-9cc0-e77929b53369 None None] MFA Rules not processed for user `b90f8980596849debe472615e7030874`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:36.678820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e2a1eb1f-fc69-45cf-9cc0-e77929b53369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:36.687979 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 901/1801] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:36 2022] PUT /identity/v3/projects/d7206977944f4875a92b6ac6db3f1b58/users/905dff5f830546289a6dcb1d12004c92/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:36.689219 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3692b0-1245-4351-b844-c3b7543fd14c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.689591 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3692b0-1245-4351-b844-c3b7543fd14c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.689873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3692b0-1245-4351-b844-c3b7543fd14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:36.708986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-dd3692b0-1245-4351-b844-c3b7543fd14c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:36.710222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-dd3692b0-1245-4351-b844-c3b7543fd14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:36.750317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 901/1802] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:36.755043 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1eb1f-fc69-45cf-9cc0-e77929b53369 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:36.766562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a1eb1f-fc69-45cf-9cc0-e77929b53369 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:36.779068 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 902/1803] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:36.782302 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3692b0-1245-4351-b844-c3b7543fd14c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:36.784097 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-dd3692b0-1245-4351-b844-c3b7543fd14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:36.785957 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:36.825505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:36.846566 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc38225-c969-403b-b160-ef0e71de353e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:36.847225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc38225-c969-403b-b160-ef0e71de353e admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.847521 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc38225-c969-403b-b160-ef0e71de353e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.847813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc38225-c969-403b-b160-ef0e71de353e admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:36.848439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc38225-c969-403b-b160-ef0e71de353e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:36.851255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc38225-c969-403b-b160-ef0e71de353e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:36.855501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3692b0-1245-4351-b844-c3b7543fd14c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:36.862090 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bfa47-8f1f-4561-bc2d-c7bac5427bd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:36.863108 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bfa47-8f1f-4561-bc2d-c7bac5427bd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.863594 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bfa47-8f1f-4561-bc2d-c7bac5427bd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.864080 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bfa47-8f1f-4561-bc2d-c7bac5427bd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:36.867413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 902/1804] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:36.868476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc38225-c969-403b-b160-ef0e71de353e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:36.884160 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc38225-c969-403b-b160-ef0e71de353e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:36.890644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd011ef-8dd4-4376-994c-c118aeede27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:36.891261 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd011ef-8dd4-4376-994c-c118aeede27e admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.891542 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd011ef-8dd4-4376-994c-c118aeede27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.891812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd011ef-8dd4-4376-994c-c118aeede27e admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:36.892457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd011ef-8dd4-4376-994c-c118aeede27e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:36.892758 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd011ef-8dd4-4376-994c-c118aeede27e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:36.894535 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd011ef-8dd4-4376-994c-c118aeede27e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:36.910861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 903/1805] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:12:36.910861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd011ef-8dd4-4376-994c-c118aeede27e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:36.918541 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd011ef-8dd4-4376-994c-c118aeede27e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:36.923299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79642cbb-1090-4a58-ae98-c577bd502fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:36.923961 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79642cbb-1090-4a58-ae98-c577bd502fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.924234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79642cbb-1090-4a58-ae98-c577bd502fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.924506 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79642cbb-1090-4a58-ae98-c577bd502fbb admin admin] PATH_INFO: `/v3/projects/d7206977944f4875a92b6ac6db3f1b58/users/905dff5f830546289a6dcb1d12004c92/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:36.926523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79642cbb-1090-4a58-ae98-c577bd502fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7206977944f4875a92b6ac6db3f1b58, user_id=905dff5f830546289a6dcb1d12004c92, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:36.929155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79642cbb-1090-4a58-ae98-c577bd502fbb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:36.942154 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 904/1806] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:36 2022] PUT /identity/v3/projects/d7206977944f4875a92b6ac6db3f1b58/users/905dff5f830546289a6dcb1d12004c92/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:36.943128 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79642cbb-1090-4a58-ae98-c577bd502fbb admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:36.943288 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6bfa47-8f1f-4561-bc2d-c7bac5427bd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:36.945484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6bfa47-8f1f-4561-bc2d-c7bac5427bd5 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:36.956698 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 903/1807] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:36 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:36.958132 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf55cdd-7253-4cc6-8bfb-063eb2969e21 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:36.958571 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf55cdd-7253-4cc6-8bfb-063eb2969e21 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:36.958918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf55cdd-7253-4cc6-8bfb-063eb2969e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:37.014264 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0cf55cdd-7253-4cc6-8bfb-063eb2969e21 None None] MFA Rules not processed for user `905dff5f830546289a6dcb1d12004c92`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:37.021130 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0cf55cdd-7253-4cc6-8bfb-063eb2969e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:37.046952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79642cbb-1090-4a58-ae98-c577bd502fbb admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:37.052700 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bd0cc-8315-4789-a154-a0970e8b2e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:37.053342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bd0cc-8315-4789-a154-a0970e8b2e2c admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:37.053633 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bd0cc-8315-4789-a154-a0970e8b2e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:37.053906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916bd0cc-8315-4789-a154-a0970e8b2e2c admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:37.054649 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bd0cc-8315-4789-a154-a0970e8b2e2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:37.057137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916bd0cc-8315-4789-a154-a0970e8b2e2c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:37.087622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 905/1808] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:36 2022] POST /identity/v3/projects => generated 399 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:37.088599 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bd0cc-8315-4789-a154-a0970e8b2e2c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:37.090140 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-916bd0cc-8315-4789-a154-a0970e8b2e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:37.105540 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 904/1809] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:37.106618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf55cdd-7253-4cc6-8bfb-063eb2969e21 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:37.118319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf55cdd-7253-4cc6-8bfb-063eb2969e21 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:37.160260 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916bd0cc-8315-4789-a154-a0970e8b2e2c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:37.170763 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366891f-f6d5-4d1c-bbd6-341b73851afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:37.171638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366891f-f6d5-4d1c-bbd6-341b73851afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:37.171938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366891f-f6d5-4d1c-bbd6-341b73851afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:37.172311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366891f-f6d5-4d1c-bbd6-341b73851afd admin admin] PATH_INFO: `/v3/users/d9f00d48e9734b67be5e9b474bf40461` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:37.173800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366891f-f6d5-4d1c-bbd6-341b73851afd admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9f00d48e9734b67be5e9b474bf40461)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:37.176006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366891f-f6d5-4d1c-bbd6-341b73851afd admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:37.193293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8563060-289c-45ac-b25c-e925a93d3e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:37.194135 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8563060-289c-45ac-b25c-e925a93d3e8f admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:37.194429 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8563060-289c-45ac-b25c-e925a93d3e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:37.194781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8563060-289c-45ac-b25c-e925a93d3e8f admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:37.196167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8563060-289c-45ac-b25c-e925a93d3e8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:37.198358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8563060-289c-45ac-b25c-e925a93d3e8f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:37.222559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:37.240564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:37.245251 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 905/1810] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:37 2022] POST /identity/v3/users => generated 300 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:37.245251 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8563060-289c-45ac-b25c-e925a93d3e8f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:37.295150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e366891f-f6d5-4d1c-bbd6-341b73851afd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9f00d48e9734b67be5e9b474bf40461'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:37.303748 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e366891f-f6d5-4d1c-bbd6-341b73851afd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9f00d48e9734b67be5e9b474bf40461'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:37.322905 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e366891f-f6d5-4d1c-bbd6-341b73851afd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9f00d48e9734b67be5e9b474bf40461'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:37.326227 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e366891f-f6d5-4d1c-bbd6-341b73851afd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9f00d48e9734b67be5e9b474bf40461'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:37.326887 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e366891f-f6d5-4d1c-bbd6-341b73851afd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9f00d48e9734b67be5e9b474bf40461'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:37.332603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8563060-289c-45ac-b25c-e925a93d3e8f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:37.340052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d32ea-cd91-4c78-9cf6-1ae7a327e52c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:37.340796 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d32ea-cd91-4c78-9cf6-1ae7a327e52c service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:37.341249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d32ea-cd91-4c78-9cf6-1ae7a327e52c service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:37.341475 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d32ea-cd91-4c78-9cf6-1ae7a327e52c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:37.343829 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-696d32ea-cd91-4c78-9cf6-1ae7a327e52c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:37.352912 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 906/1811] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:37 2022] DELETE /identity/v3/users/d9f00d48e9734b67be5e9b474bf40461 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:37.354968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366891f-f6d5-4d1c-bbd6-341b73851afd admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:37.356760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e366891f-f6d5-4d1c-bbd6-341b73851afd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:37.426453 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d32ea-cd91-4c78-9cf6-1ae7a327e52c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:37.428096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d32ea-cd91-4c78-9cf6-1ae7a327e52c service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:37.436575 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366891f-f6d5-4d1c-bbd6-341b73851afd admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:37.438108 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 906/1812] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:37.439307 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65968f59-94f6-4367-8aca-ff3c1674fd4c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:37.439570 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65968f59-94f6-4367-8aca-ff3c1674fd4c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:37.439840 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65968f59-94f6-4367-8aca-ff3c1674fd4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:37.441655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f23d80-6aff-4881-b6a7-f66ac7213655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:37.442286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f23d80-6aff-4881-b6a7-f66ac7213655 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:37.442562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f23d80-6aff-4881-b6a7-f66ac7213655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:37.442840 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f23d80-6aff-4881-b6a7-f66ac7213655 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:37.443467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f23d80-6aff-4881-b6a7-f66ac7213655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:37.445045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f23d80-6aff-4881-b6a7-f66ac7213655 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:37.459943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 907/1813] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:37.462446 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-65968f59-94f6-4367-8aca-ff3c1674fd4c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:37.463626 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-65968f59-94f6-4367-8aca-ff3c1674fd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:37.464841 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f23d80-6aff-4881-b6a7-f66ac7213655 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:37.473363 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 907/1814] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:37.477970 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f23d80-6aff-4881-b6a7-f66ac7213655 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:37.483677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65968f59-94f6-4367-8aca-ff3c1674fd4c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:37.483983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ddc15-91b9-49ac-9fab-71018cf7cf62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:37.485075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ddc15-91b9-49ac-9fab-71018cf7cf62 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:37.485390 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ddc15-91b9-49ac-9fab-71018cf7cf62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:37.485663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464ddc15-91b9-49ac-9fab-71018cf7cf62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:37.486298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ddc15-91b9-49ac-9fab-71018cf7cf62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:37.486404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-65968f59-94f6-4367-8aca-ff3c1674fd4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:37.487893 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464ddc15-91b9-49ac-9fab-71018cf7cf62 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:37.509237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 908/1815] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:37.512334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ddc15-91b9-49ac-9fab-71018cf7cf62 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:37.532242 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464ddc15-91b9-49ac-9fab-71018cf7cf62 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:37.542505 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c85f7d-c796-4589-99d3-07890e445bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:37.543173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c85f7d-c796-4589-99d3-07890e445bbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:37.543433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c85f7d-c796-4589-99d3-07890e445bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:37.543708 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c85f7d-c796-4589-99d3-07890e445bbb admin admin] PATH_INFO: `/v3/projects/4a6741ed6c3348f499654726f04d0b4a/users/bd81c673b97648ba91fcdceddf72a828/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:37.570348 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c85f7d-c796-4589-99d3-07890e445bbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a6741ed6c3348f499654726f04d0b4a, user_id=bd81c673b97648ba91fcdceddf72a828, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:37.573800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c85f7d-c796-4589-99d3-07890e445bbb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:37.596312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 909/1816] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:37 2022] PUT /identity/v3/projects/4a6741ed6c3348f499654726f04d0b4a/users/bd81c673b97648ba91fcdceddf72a828/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:37.597217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c85f7d-c796-4589-99d3-07890e445bbb admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:37.606173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-72c85f7d-c796-4589-99d3-07890e445bbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:37.627399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65968f59-94f6-4367-8aca-ff3c1674fd4c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:37.707483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec992f2-4a1d-48aa-8ae7-9343d02003b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:37.708022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c85f7d-c796-4589-99d3-07890e445bbb admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:37.708531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec992f2-4a1d-48aa-8ae7-9343d02003b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:37.708531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec992f2-4a1d-48aa-8ae7-9343d02003b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:37.708698 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec992f2-4a1d-48aa-8ae7-9343d02003b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:37.711884 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d3a0c-1dbc-44c3-8741-15031aa8cf5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:37.713112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d3a0c-1dbc-44c3-8741-15031aa8cf5b service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:37.713448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d3a0c-1dbc-44c3-8741-15031aa8cf5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:37.713743 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6d3a0c-1dbc-44c3-8741-15031aa8cf5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:37.715697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5d6d3a0c-1dbc-44c3-8741-15031aa8cf5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:37.719485 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:37.733018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:37.818014 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec992f2-4a1d-48aa-8ae7-9343d02003b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:37.819510 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec992f2-4a1d-48aa-8ae7-9343d02003b2 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:37.827813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d3a0c-1dbc-44c3-8741-15031aa8cf5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:37.829304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6d3a0c-1dbc-44c3-8741-15031aa8cf5b service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:37.830556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 908/1817] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:12:37.831918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565b011-df9a-46cc-9bb7-ab9960e70526 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:37.832211 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565b011-df9a-46cc-9bb7-ab9960e70526 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:37.832490 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b565b011-df9a-46cc-9bb7-ab9960e70526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:37.840217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 910/1818] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:37.841117 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d3a0c-1dbc-44c3-8741-15031aa8cf5b service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:37.842459 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5d6d3a0c-1dbc-44c3-8741-15031aa8cf5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:37.857210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6d3a0c-1dbc-44c3-8741-15031aa8cf5b service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:37.863120 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba492b0f-449b-4ce1-b077-3210ba4d4178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:37.863837 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba492b0f-449b-4ce1-b077-3210ba4d4178 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:37.864189 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba492b0f-449b-4ce1-b077-3210ba4d4178 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:37.864554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba492b0f-449b-4ce1-b077-3210ba4d4178 admin admin] PATH_INFO: `/v3/users/cd659521cae04153ae616776358bba66` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:37.866038 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba492b0f-449b-4ce1-b077-3210ba4d4178 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd659521cae04153ae616776358bba66)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:37.868091 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba492b0f-449b-4ce1-b077-3210ba4d4178 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:37.880646 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b565b011-df9a-46cc-9bb7-ab9960e70526 None None] MFA Rules not processed for user `bd81c673b97648ba91fcdceddf72a828`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:37.883664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b565b011-df9a-46cc-9bb7-ab9960e70526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:37.986403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ba492b0f-449b-4ce1-b077-3210ba4d4178 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd659521cae04153ae616776358bba66'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:37.999118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 909/1819] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:37 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:38.003715 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565b011-df9a-46cc-9bb7-ab9960e70526 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:38.004077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ba492b0f-449b-4ce1-b077-3210ba4d4178 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd659521cae04153ae616776358bba66'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:38.016556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b565b011-df9a-46cc-9bb7-ab9960e70526 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:38.022058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ba492b0f-449b-4ce1-b077-3210ba4d4178 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd659521cae04153ae616776358bba66'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:38.022645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8212c-a0fd-4ace-98a9-125f6d51bc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:38.022885 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ba492b0f-449b-4ce1-b077-3210ba4d4178 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd659521cae04153ae616776358bba66'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:38.023235 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8212c-a0fd-4ace-98a9-125f6d51bc12 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:38.023516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8212c-a0fd-4ace-98a9-125f6d51bc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:38.023630 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ba492b0f-449b-4ce1-b077-3210ba4d4178 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd659521cae04153ae616776358bba66'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:38.023794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a8212c-a0fd-4ace-98a9-125f6d51bc12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:38.024410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8212c-a0fd-4ace-98a9-125f6d51bc12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:38.028164 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a8212c-a0fd-4ace-98a9-125f6d51bc12 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:38.068991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 911/1820] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:37 2022] DELETE /identity/v3/users/cd659521cae04153ae616776358bba66 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:38.073901 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c8a27-c510-4f06-8f86-511aa203c14c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:38.074270 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c8a27-c510-4f06-8f86-511aa203c14c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:38.074616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777c8a27-c510-4f06-8f86-511aa203c14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:38.079918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 910/1821] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:38 2022] POST /identity/v3/projects => generated 401 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:38.080931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8212c-a0fd-4ace-98a9-125f6d51bc12 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:38.084244 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-57a8212c-a0fd-4ace-98a9-125f6d51bc12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:38.163478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-777c8a27-c510-4f06-8f86-511aa203c14c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:38.164757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-777c8a27-c510-4f06-8f86-511aa203c14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:38.250533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a8212c-a0fd-4ace-98a9-125f6d51bc12 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:38.266374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce6c44-cde8-4289-86ad-335ecd0d3ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:38.267114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce6c44-cde8-4289-86ad-335ecd0d3ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:38.267466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce6c44-cde8-4289-86ad-335ecd0d3ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:38.267809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ce6c44-cde8-4289-86ad-335ecd0d3ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:38.268681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce6c44-cde8-4289-86ad-335ecd0d3ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:38.271564 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ce6c44-cde8-4289-86ad-335ecd0d3ae4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:38.316734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 911/1822] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:38 2022] POST /identity/v3/users => generated 307 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:38.318347 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce6c44-cde8-4289-86ad-335ecd0d3ae4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:38.325933 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:38.344498 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ce6c44-cde8-4289-86ad-335ecd0d3ae4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:38.354139 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb545281-7835-40d2-ac09-ed784b6abd1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:38.355310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545281-7835-40d2-ac09-ed784b6abd1b admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:38.356011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545281-7835-40d2-ac09-ed784b6abd1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:38.356589 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545281-7835-40d2-ac09-ed784b6abd1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:38.360609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb545281-7835-40d2-ac09-ed784b6abd1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:38.363904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb545281-7835-40d2-ac09-ed784b6abd1b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:38.372401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 912/1823] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:38.377043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 912/1824] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:38.382143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c8a27-c510-4f06-8f86-511aa203c14c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:38.394130 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb545281-7835-40d2-ac09-ed784b6abd1b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:38.396992 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-eb545281-7835-40d2-ac09-ed784b6abd1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:38.403474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777c8a27-c510-4f06-8f86-511aa203c14c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:38.421574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea9051-a329-44c6-9e62-e294a8658581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:38.423587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea9051-a329-44c6-9e62-e294a8658581 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:38.423864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea9051-a329-44c6-9e62-e294a8658581 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:38.424736 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aea9051-a329-44c6-9e62-e294a8658581 admin admin] PATH_INFO: `/v3/projects/e5712a809c2d4b5bbd81bd45e9d51074/users/52d68c02d7b74897add3e84e6c468463/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:38.437895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb545281-7835-40d2-ac09-ed784b6abd1b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:38.449533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3fb71-9283-445c-aa90-30049874eb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:38.450414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3fb71-9283-445c-aa90-30049874eb87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:38.452693 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3fb71-9283-445c-aa90-30049874eb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:38.453152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c3fb71-9283-445c-aa90-30049874eb87 admin admin] PATH_INFO: `/v3/projects/6b1888d89fb2437fa3f4dc88d696ebfe` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:38.457354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3fb71-9283-445c-aa90-30049874eb87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b1888d89fb2437fa3f4dc88d696ebfe)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:38.463105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c3fb71-9283-445c-aa90-30049874eb87 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:38.471025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea9051-a329-44c6-9e62-e294a8658581 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5712a809c2d4b5bbd81bd45e9d51074, user_id=52d68c02d7b74897add3e84e6c468463, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:38.477144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aea9051-a329-44c6-9e62-e294a8658581 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:38.521529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:38.535130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 913/1825] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:38 2022] PUT /identity/v3/projects/e5712a809c2d4b5bbd81bd45e9d51074/users/52d68c02d7b74897add3e84e6c468463/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:12:38.542619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea9051-a329-44c6-9e62-e294a8658581 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:38.545215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6aea9051-a329-44c6-9e62-e294a8658581 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:38.626280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-91c3fb71-9283-445c-aa90-30049874eb87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b1888d89fb2437fa3f4dc88d696ebfe', 'request_id': 'req-91c3fb71-9283-445c-aa90-30049874eb87'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:38.641099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aea9051-a329-44c6-9e62-e294a8658581 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:38.643660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-91c3fb71-9283-445c-aa90-30049874eb87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b1888d89fb2437fa3f4dc88d696ebfe', 'request_id': 'req-91c3fb71-9283-445c-aa90-30049874eb87'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:38.658899 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-91c3fb71-9283-445c-aa90-30049874eb87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b1888d89fb2437fa3f4dc88d696ebfe', 'request_id': 'req-91c3fb71-9283-445c-aa90-30049874eb87'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:38.683382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-91c3fb71-9283-445c-aa90-30049874eb87 admin admin] The token cache is being invalidate because project 6b1888d89fb2437fa3f4dc88d696ebfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:38.683673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-91c3fb71-9283-445c-aa90-30049874eb87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:38.685597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-91c3fb71-9283-445c-aa90-30049874eb87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:38.689460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 913/1826] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:38 2022] DELETE /identity/v3/projects/6b1888d89fb2437fa3f4dc88d696ebfe => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:38.691783 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3fb71-9283-445c-aa90-30049874eb87 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:38.694280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-91c3fb71-9283-445c-aa90-30049874eb87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:38.703267 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a71541-e099-4218-b92c-8e197d5f5aa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:38.704032 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a71541-e099-4218-b92c-8e197d5f5aa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:38.704470 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a71541-e099-4218-b92c-8e197d5f5aa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:38.704858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a71541-e099-4218-b92c-8e197d5f5aa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:38.707310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-14a71541-e099-4218-b92c-8e197d5f5aa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:38.796110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a71541-e099-4218-b92c-8e197d5f5aa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:38.798097 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a71541-e099-4218-b92c-8e197d5f5aa2 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:38.811243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 914/1827] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:38.812430 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a71541-e099-4218-b92c-8e197d5f5aa2 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:38.812734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c3fb71-9283-445c-aa90-30049874eb87 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:38.814235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-14a71541-e099-4218-b92c-8e197d5f5aa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:38.818548 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1276cf-f4d4-4654-9b82-9d1f48c2d9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:38.819271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1276cf-f4d4-4654-9b82-9d1f48c2d9ff admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:38.819782 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1276cf-f4d4-4654-9b82-9d1f48c2d9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:38.820255 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1276cf-f4d4-4654-9b82-9d1f48c2d9ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:38.821075 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1276cf-f4d4-4654-9b82-9d1f48c2d9ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:38.823202 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1276cf-f4d4-4654-9b82-9d1f48c2d9ff admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:38.830457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a71541-e099-4218-b92c-8e197d5f5aa2 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:38.836239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51153ada-df7f-469a-b8f0-06c14f8a2dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:38.837088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51153ada-df7f-469a-b8f0-06c14f8a2dd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:38.837614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51153ada-df7f-469a-b8f0-06c14f8a2dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:38.838075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51153ada-df7f-469a-b8f0-06c14f8a2dd9 admin admin] PATH_INFO: `/v3/users/b90f8980596849debe472615e7030874` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:38.838756 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 914/1828] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:38 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:38.842517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51153ada-df7f-469a-b8f0-06c14f8a2dd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b90f8980596849debe472615e7030874)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:38.844639 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51153ada-df7f-469a-b8f0-06c14f8a2dd9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:38.852780 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1276cf-f4d4-4654-9b82-9d1f48c2d9ff admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:38.877559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1276cf-f4d4-4654-9b82-9d1f48c2d9ff admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:38.905500 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a870f8-9e61-4884-a905-4d89c831bd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:38.906274 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a870f8-9e61-4884-a905-4d89c831bd3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:38.906560 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a870f8-9e61-4884-a905-4d89c831bd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:38.906918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a870f8-9e61-4884-a905-4d89c831bd3b admin admin] PATH_INFO: `/v3/projects/e5712a809c2d4b5bbd81bd45e9d51074/users/52d68c02d7b74897add3e84e6c468463/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:38.909064 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a870f8-9e61-4884-a905-4d89c831bd3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5712a809c2d4b5bbd81bd45e9d51074, user_id=52d68c02d7b74897add3e84e6c468463, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:38.912212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a870f8-9e61-4884-a905-4d89c831bd3b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:38.940749 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 915/1829] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:38 2022] PUT /identity/v3/projects/e5712a809c2d4b5bbd81bd45e9d51074/users/52d68c02d7b74897add3e84e6c468463/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:12:38.951278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe3f60-6c03-4ce5-b6e4-3cb298e47999 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:38.951610 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe3f60-6c03-4ce5-b6e4-3cb298e47999 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:38.951843 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe3f60-6c03-4ce5-b6e4-3cb298e47999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:38.987612 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-80fe3f60-6c03-4ce5-b6e4-3cb298e47999 None None] MFA Rules not processed for user `52d68c02d7b74897add3e84e6c468463`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:38.989128 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-80fe3f60-6c03-4ce5-b6e4-3cb298e47999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:39.007904 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-51153ada-df7f-469a-b8f0-06c14f8a2dd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b90f8980596849debe472615e7030874'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:39.019338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-51153ada-df7f-469a-b8f0-06c14f8a2dd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b90f8980596849debe472615e7030874'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:39.022602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:39.045404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-51153ada-df7f-469a-b8f0-06c14f8a2dd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b90f8980596849debe472615e7030874'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:39.046317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-51153ada-df7f-469a-b8f0-06c14f8a2dd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b90f8980596849debe472615e7030874'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:39.047312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-51153ada-df7f-469a-b8f0-06c14f8a2dd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b90f8980596849debe472615e7030874'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:39.061151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:39.087801 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 915/1830] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:38 2022] DELETE /identity/v3/users/b90f8980596849debe472615e7030874 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:39.110866 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 916/1831] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:38 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:39.124315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51153ada-df7f-469a-b8f0-06c14f8a2dd9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:39.126392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-51153ada-df7f-469a-b8f0-06c14f8a2dd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:39.264681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51153ada-df7f-469a-b8f0-06c14f8a2dd9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:39.280190 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97e785-0486-41de-b8b3-971487a7ebc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:39.281129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97e785-0486-41de-b8b3-971487a7ebc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:39.281595 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97e785-0486-41de-b8b3-971487a7ebc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:39.282023 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97e785-0486-41de-b8b3-971487a7ebc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:39.284900 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe3f60-6c03-4ce5-b6e4-3cb298e47999 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:39.289175 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-80fe3f60-6c03-4ce5-b6e4-3cb298e47999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:39.452017 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe3f60-6c03-4ce5-b6e4-3cb298e47999 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:39.468373 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97e785-0486-41de-b8b3-971487a7ebc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:39.471241 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97e785-0486-41de-b8b3-971487a7ebc4 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:39.503014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 916/1832] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:39 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:39.615772 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b35ca-5e72-4511-bb78-70580ef11454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:39.616441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b35ca-5e72-4511-bb78-70580ef11454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:39.617028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b35ca-5e72-4511-bb78-70580ef11454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:39.617212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139b35ca-5e72-4511-bb78-70580ef11454 admin admin] PATH_INFO: `/v3/projects/81bcc34e27e8473da924e2bed9783eb9` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:39.618696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b35ca-5e72-4511-bb78-70580ef11454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81bcc34e27e8473da924e2bed9783eb9)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:39.622880 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97e785-0486-41de-b8b3-971487a7ebc4 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:39.624479 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139b35ca-5e72-4511-bb78-70580ef11454 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:39.627699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6f97e785-0486-41de-b8b3-971487a7ebc4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:39.663848 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97e785-0486-41de-b8b3-971487a7ebc4 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:39.703103 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:39.706754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:39.707816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:39.708108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2 admin admin] PATH_INFO: `/v3/projects/0d22285f09b94145bab87e8727008288` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:39.709715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d22285f09b94145bab87e8727008288)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:39.716233 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:39.738456 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-139b35ca-5e72-4511-bb78-70580ef11454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81bcc34e27e8473da924e2bed9783eb9', 'request_id': 'req-139b35ca-5e72-4511-bb78-70580ef11454'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:39.750366 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-139b35ca-5e72-4511-bb78-70580ef11454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81bcc34e27e8473da924e2bed9783eb9', 'request_id': 'req-139b35ca-5e72-4511-bb78-70580ef11454'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:39.767362 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-139b35ca-5e72-4511-bb78-70580ef11454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81bcc34e27e8473da924e2bed9783eb9', 'request_id': 'req-139b35ca-5e72-4511-bb78-70580ef11454'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:39.804462 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-139b35ca-5e72-4511-bb78-70580ef11454 admin admin] The token cache is being invalidate because project 81bcc34e27e8473da924e2bed9783eb9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:39.804875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-139b35ca-5e72-4511-bb78-70580ef11454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:39.808489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-139b35ca-5e72-4511-bb78-70580ef11454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:39.810703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 917/1833] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:39 2022] DELETE /identity/v3/projects/81bcc34e27e8473da924e2bed9783eb9 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:12:39.815159 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b35ca-5e72-4511-bb78-70580ef11454 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:39.816892 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-139b35ca-5e72-4511-bb78-70580ef11454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:39.864717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d22285f09b94145bab87e8727008288', 'request_id': 'req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:39.868387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:39.880594 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d22285f09b94145bab87e8727008288', 'request_id': 'req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:39.888780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d22285f09b94145bab87e8727008288', 'request_id': 'req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:39.916581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2 admin admin] The token cache is being invalidate because project 0d22285f09b94145bab87e8727008288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:39.917122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:39.920808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:39.932729 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 917/1834] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:39 2022] DELETE /identity/v3/projects/0d22285f09b94145bab87e8727008288 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:39.934265 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:39.936656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:39.973302 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:39.980451 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139b35ca-5e72-4511-bb78-70580ef11454 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:40.063559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ade60b-ef5e-494a-b3ca-8d52ab940fd2 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:40.075108 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7237a1-3835-43c3-9807-6048bfdcef39 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:40.075323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35e8adb-0069-4ab6-a6f2-f892e9881467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:40.075878 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7237a1-3835-43c3-9807-6048bfdcef39 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.076143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e8adb-0069-4ab6-a6f2-f892e9881467 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.076241 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7237a1-3835-43c3-9807-6048bfdcef39 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.076411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7237a1-3835-43c3-9807-6048bfdcef39 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.076941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e8adb-0069-4ab6-a6f2-f892e9881467 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.077312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35e8adb-0069-4ab6-a6f2-f892e9881467 admin admin] PATH_INFO: `/v3/users/905dff5f830546289a6dcb1d12004c92` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.078186 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ff7237a1-3835-43c3-9807-6048bfdcef39 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:40.078600 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e8adb-0069-4ab6-a6f2-f892e9881467 admin admin] RBAC: Authorizing `identity:delete_user(user_id=905dff5f830546289a6dcb1d12004c92)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:40.080663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35e8adb-0069-4ab6-a6f2-f892e9881467 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:40.133057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e35e8adb-0069-4ab6-a6f2-f892e9881467 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '905dff5f830546289a6dcb1d12004c92'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:40.141245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e35e8adb-0069-4ab6-a6f2-f892e9881467 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '905dff5f830546289a6dcb1d12004c92'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:40.142643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7237a1-3835-43c3-9807-6048bfdcef39 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:40.144068 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7237a1-3835-43c3-9807-6048bfdcef39 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:40.153818 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 918/1835] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:39 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:40.154753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e35e8adb-0069-4ab6-a6f2-f892e9881467 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '905dff5f830546289a6dcb1d12004c92'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:40.155020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670715b-0ebe-407f-845d-1fff590d97cc None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.155293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670715b-0ebe-407f-845d-1fff590d97cc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.155459 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e35e8adb-0069-4ab6-a6f2-f892e9881467 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '905dff5f830546289a6dcb1d12004c92'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:40.155567 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5670715b-0ebe-407f-845d-1fff590d97cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.156175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e35e8adb-0069-4ab6-a6f2-f892e9881467 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '905dff5f830546289a6dcb1d12004c92'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:40.178562 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-5670715b-0ebe-407f-845d-1fff590d97cc None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:40.178830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 918/1836] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:39 2022] DELETE /identity/v3/users/905dff5f830546289a6dcb1d12004c92 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:40.179651 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5670715b-0ebe-407f-845d-1fff590d97cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:40.275057 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 919/1837] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:40.283917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d25ee-45e3-4932-aafc-31ada4950c4c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.285107 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d25ee-45e3-4932-aafc-31ada4950c4c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.285801 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d25ee-45e3-4932-aafc-31ada4950c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.316147 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3b5d25ee-45e3-4932-aafc-31ada4950c4c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:40.317417 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3b5d25ee-45e3-4932-aafc-31ada4950c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:40.330806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 919/1838] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:40.332537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670715b-0ebe-407f-845d-1fff590d97cc None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:40.345592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5670715b-0ebe-407f-845d-1fff590d97cc None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:40.360030 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96593a-615a-4e54-add4-0878bc7d4c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:40.360666 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96593a-615a-4e54-add4-0878bc7d4c00 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.360948 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96593a-615a-4e54-add4-0878bc7d4c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.361247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c96593a-615a-4e54-add4-0878bc7d4c00 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.361957 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96593a-615a-4e54-add4-0878bc7d4c00 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:40.362267 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96593a-615a-4e54-add4-0878bc7d4c00 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:40.363879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c96593a-615a-4e54-add4-0878bc7d4c00 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:40.382804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 920/1839] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:12:40.387238 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d25ee-45e3-4932-aafc-31ada4950c4c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:40.398103 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d25ee-45e3-4932-aafc-31ada4950c4c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:40.405834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567fd20-80f6-4569-a932-a0f6447a3cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:40.406903 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567fd20-80f6-4569-a932-a0f6447a3cba admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.407186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567fd20-80f6-4569-a932-a0f6447a3cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.407460 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567fd20-80f6-4569-a932-a0f6447a3cba admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.408107 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567fd20-80f6-4569-a932-a0f6447a3cba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:40.410280 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567fd20-80f6-4569-a932-a0f6447a3cba admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:40.449241 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 920/1840] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:40 2022] POST /identity/v3/projects => generated 379 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:40.452147 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96593a-615a-4e54-add4-0878bc7d4c00 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:40.473004 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c96593a-615a-4e54-add4-0878bc7d4c00 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:40.491565 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:40.557653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567fd20-80f6-4569-a932-a0f6447a3cba admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:40.559410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f567fd20-80f6-4569-a932-a0f6447a3cba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:40.579013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6545f8-909d-48df-9d1c-2251277a03a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:40.580015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6545f8-909d-48df-9d1c-2251277a03a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.580473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6545f8-909d-48df-9d1c-2251277a03a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.580749 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6545f8-909d-48df-9d1c-2251277a03a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.581983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6545f8-909d-48df-9d1c-2251277a03a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:40.584282 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6545f8-909d-48df-9d1c-2251277a03a6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:40.608442 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 921/1841] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:40 2022] POST /identity/v3/users => generated 290 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:40.613311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6545f8-909d-48df-9d1c-2251277a03a6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:40.624977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6545f8-909d-48df-9d1c-2251277a03a6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:40.629633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599fe1f-4171-498c-b74e-93b7d12559c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:40.630239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599fe1f-4171-498c-b74e-93b7d12559c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.630542 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599fe1f-4171-498c-b74e-93b7d12559c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.630810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6599fe1f-4171-498c-b74e-93b7d12559c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.631444 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599fe1f-4171-498c-b74e-93b7d12559c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:40.633042 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6599fe1f-4171-498c-b74e-93b7d12559c3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:40.633737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567fd20-80f6-4569-a932-a0f6447a3cba admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:40.638537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c566a5c-c23c-4d66-8e44-b8821a1e0dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:40.639097 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c566a5c-c23c-4d66-8e44-b8821a1e0dc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.639373 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c566a5c-c23c-4d66-8e44-b8821a1e0dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.639643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c566a5c-c23c-4d66-8e44-b8821a1e0dc1 admin admin] PATH_INFO: `/v3/projects/d7206977944f4875a92b6ac6db3f1b58` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.640762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c566a5c-c23c-4d66-8e44-b8821a1e0dc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7206977944f4875a92b6ac6db3f1b58)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:40.642731 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c566a5c-c23c-4d66-8e44-b8821a1e0dc1 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:40.644349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:40.644715 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 922/1842] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:40.648785 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599fe1f-4171-498c-b74e-93b7d12559c3 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:40.660711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6599fe1f-4171-498c-b74e-93b7d12559c3 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:40.665349 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab6fe7-e445-4592-8bf7-bae7e5e2006e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:40.666000 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab6fe7-e445-4592-8bf7-bae7e5e2006e admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.666286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab6fe7-e445-4592-8bf7-bae7e5e2006e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.667065 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab6fe7-e445-4592-8bf7-bae7e5e2006e admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.667699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab6fe7-e445-4592-8bf7-bae7e5e2006e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:40.669306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab6fe7-e445-4592-8bf7-bae7e5e2006e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:40.679389 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 923/1843] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:12:40.683476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab6fe7-e445-4592-8bf7-bae7e5e2006e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:40.692984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab6fe7-e445-4592-8bf7-bae7e5e2006e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:40.694465 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3c566a5c-c23c-4d66-8e44-b8821a1e0dc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7206977944f4875a92b6ac6db3f1b58', 'request_id': 'req-3c566a5c-c23c-4d66-8e44-b8821a1e0dc1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:40.702525 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3c566a5c-c23c-4d66-8e44-b8821a1e0dc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7206977944f4875a92b6ac6db3f1b58', 'request_id': 'req-3c566a5c-c23c-4d66-8e44-b8821a1e0dc1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:40.708334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3c566a5c-c23c-4d66-8e44-b8821a1e0dc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7206977944f4875a92b6ac6db3f1b58', 'request_id': 'req-3c566a5c-c23c-4d66-8e44-b8821a1e0dc1'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:40.720078 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3c566a5c-c23c-4d66-8e44-b8821a1e0dc1 admin admin] The token cache is being invalidate because project d7206977944f4875a92b6ac6db3f1b58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:40.720362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3c566a5c-c23c-4d66-8e44-b8821a1e0dc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:40.721051 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3c566a5c-c23c-4d66-8e44-b8821a1e0dc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:40.726457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 921/1844] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:40 2022] DELETE /identity/v3/projects/d7206977944f4875a92b6ac6db3f1b58 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:40.727877 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f824f3f-aa71-48d3-9a65-feccd2cff7ae None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.728807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f824f3f-aa71-48d3-9a65-feccd2cff7ae None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.729080 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f824f3f-aa71-48d3-9a65-feccd2cff7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.749303 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2f824f3f-aa71-48d3-9a65-feccd2cff7ae None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:40.750503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2f824f3f-aa71-48d3-9a65-feccd2cff7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:40.752123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ccd669-1dc9-4e1b-a739-d99ef774ff19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:40.752727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccd669-1dc9-4e1b-a739-d99ef774ff19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.753010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccd669-1dc9-4e1b-a739-d99ef774ff19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.753322 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ccd669-1dc9-4e1b-a739-d99ef774ff19 admin admin] PATH_INFO: `/v3/projects/86c9d54d1be7403aa58dcd72c9fd41af/users/de13e3506df84492af284766b061ed4e/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.760733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 922/1845] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:40.768198 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253e7b5-1a70-4e59-a823-f439dfaff8fb None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.768481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253e7b5-1a70-4e59-a823-f439dfaff8fb None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.768753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253e7b5-1a70-4e59-a823-f439dfaff8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.771888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccd669-1dc9-4e1b-a739-d99ef774ff19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86c9d54d1be7403aa58dcd72c9fd41af, user_id=de13e3506df84492af284766b061ed4e, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:40.774699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ccd669-1dc9-4e1b-a739-d99ef774ff19 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:40.788316 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 924/1846] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:40 2022] PUT /identity/v3/projects/86c9d54d1be7403aa58dcd72c9fd41af/users/de13e3506df84492af284766b061ed4e/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:12:40.789587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c253e7b5-1a70-4e59-a823-f439dfaff8fb None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:40.791467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c253e7b5-1a70-4e59-a823-f439dfaff8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:40.799265 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688ef61-8b12-4b89-9694-f81578fe4238 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.799543 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688ef61-8b12-4b89-9694-f81578fe4238 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.799813 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2688ef61-8b12-4b89-9694-f81578fe4238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.827403 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2688ef61-8b12-4b89-9694-f81578fe4238 None None] MFA Rules not processed for user `de13e3506df84492af284766b061ed4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:40.828644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2688ef61-8b12-4b89-9694-f81578fe4238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:40.852150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 923/1847] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:12:40.855933 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253e7b5-1a70-4e59-a823-f439dfaff8fb None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:40.867017 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253e7b5-1a70-4e59-a823-f439dfaff8fb None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:40.872953 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03dfea-2029-42ae-9c2a-495fe42aff79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:40.873574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03dfea-2029-42ae-9c2a-495fe42aff79 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.873855 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03dfea-2029-42ae-9c2a-495fe42aff79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.874125 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb03dfea-2029-42ae-9c2a-495fe42aff79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.874845 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03dfea-2029-42ae-9c2a-495fe42aff79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:40.875149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03dfea-2029-42ae-9c2a-495fe42aff79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:40.876731 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb03dfea-2029-42ae-9c2a-495fe42aff79 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:40.888831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 924/1848] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:12:40.889377 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:40.892992 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03dfea-2029-42ae-9c2a-495fe42aff79 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:40.896018 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 925/1849] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:12:40.901033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688ef61-8b12-4b89-9694-f81578fe4238 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:40.902449 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2688ef61-8b12-4b89-9694-f81578fe4238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:40.903535 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb03dfea-2029-42ae-9c2a-495fe42aff79 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:40.908115 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a71824-33a6-4401-8569-a9061e9b06bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:40.908704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a71824-33a6-4401-8569-a9061e9b06bf admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.908979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a71824-33a6-4401-8569-a9061e9b06bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.909290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a71824-33a6-4401-8569-a9061e9b06bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.909911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a71824-33a6-4401-8569-a9061e9b06bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:40.911918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a71824-33a6-4401-8569-a9061e9b06bf admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:40.915377 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2688ef61-8b12-4b89-9694-f81578fe4238 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:40.920011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f734a-603d-441e-951d-62741308fa65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:40.920604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f734a-603d-441e-951d-62741308fa65 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:40.920888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f734a-603d-441e-951d-62741308fa65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:40.921164 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f734a-603d-441e-951d-62741308fa65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:40.921851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f734a-603d-441e-951d-62741308fa65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:40.923768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f734a-603d-441e-951d-62741308fa65 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:40.940840 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 925/1850] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:40 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:12:40.945176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a71824-33a6-4401-8569-a9061e9b06bf admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:40.957021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 926/1851] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:40 2022] POST /identity/v3/projects => generated 379 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:12:40.961037 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f734a-603d-441e-951d-62741308fa65 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:41.001269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:41.018546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a71824-33a6-4401-8569-a9061e9b06bf admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:41.032750 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f734a-603d-441e-951d-62741308fa65 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:41.038849 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abbfa55-7a64-42e0-a385-d18fa21d62cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:41.040095 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbfa55-7a64-42e0-a385-d18fa21d62cd admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.040417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbfa55-7a64-42e0-a385-d18fa21d62cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.040725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abbfa55-7a64-42e0-a385-d18fa21d62cd admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.041783 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abbfa55-7a64-42e0-a385-d18fa21d62cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:41.044027 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abbfa55-7a64-42e0-a385-d18fa21d62cd admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:41.078318 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 927/1852] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:40 2022] POST /identity/v3/users => generated 296 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:12:41.083369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abbfa55-7a64-42e0-a385-d18fa21d62cd admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:41.103554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f71d6-6035-41c6-be97-3c376c393773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:41.104228 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f71d6-6035-41c6-be97-3c376c393773 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.104435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f71d6-6035-41c6-be97-3c376c393773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.104701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9f71d6-6035-41c6-be97-3c376c393773 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.105366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f71d6-6035-41c6-be97-3c376c393773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:41.107283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9f71d6-6035-41c6-be97-3c376c393773 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:41.111672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abbfa55-7a64-42e0-a385-d18fa21d62cd admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:41.116910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d2ba4-dd33-4452-a044-ef428187bea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:41.117726 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d2ba4-dd33-4452-a044-ef428187bea9 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.118377 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d2ba4-dd33-4452-a044-ef428187bea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.118796 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d2ba4-dd33-4452-a044-ef428187bea9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.119546 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d2ba4-dd33-4452-a044-ef428187bea9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:41.121936 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d2ba4-dd33-4452-a044-ef428187bea9 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:41.140880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 928/1853] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:41.146204 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d2ba4-dd33-4452-a044-ef428187bea9 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:41.147404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 926/1854] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:40 2022] POST /identity/v3/users => generated 290 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:12:41.162811 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f71d6-6035-41c6-be97-3c376c393773 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:41.168276 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d2ba4-dd33-4452-a044-ef428187bea9 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:41.177114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb1491-b379-4935-a7e6-897752fd2a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:41.177811 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb1491-b379-4935-a7e6-897752fd2a16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.178106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb1491-b379-4935-a7e6-897752fd2a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.178412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bb1491-b379-4935-a7e6-897752fd2a16 admin admin] PATH_INFO: `/v3/projects/525224b48ff8419caad88dad16a5a9b2/users/bd5e67491e07475096a81355e2b2a4d8/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.180894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9f71d6-6035-41c6-be97-3c376c393773 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:41.190173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b1332-bbf5-4858-b4f1-83869ea518a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:41.191026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b1332-bbf5-4858-b4f1-83869ea518a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.191439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b1332-bbf5-4858-b4f1-83869ea518a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.191844 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b1332-bbf5-4858-b4f1-83869ea518a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.192639 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b1332-bbf5-4858-b4f1-83869ea518a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:41.194311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3b1332-bbf5-4858-b4f1-83869ea518a4 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:41.204143 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb1491-b379-4935-a7e6-897752fd2a16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=525224b48ff8419caad88dad16a5a9b2, user_id=bd5e67491e07475096a81355e2b2a4d8, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:41.206430 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 927/1855] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:41.210158 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bb1491-b379-4935-a7e6-897752fd2a16 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:41.214116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b1332-bbf5-4858-b4f1-83869ea518a4 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:41.226439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 929/1856] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:41 2022] PUT /identity/v3/projects/525224b48ff8419caad88dad16a5a9b2/users/bd5e67491e07475096a81355e2b2a4d8/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:12:41.227597 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3b1332-bbf5-4858-b4f1-83869ea518a4 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:41.230936 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb1491-b379-4935-a7e6-897752fd2a16 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:41.297062 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33859d06-dfb7-453d-9b09-1c0439eff29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:41.297703 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33859d06-dfb7-453d-9b09-1c0439eff29a admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.297979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33859d06-dfb7-453d-9b09-1c0439eff29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.298253 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33859d06-dfb7-453d-9b09-1c0439eff29a admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.298892 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33859d06-dfb7-453d-9b09-1c0439eff29a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:41.300490 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33859d06-dfb7-453d-9b09-1c0439eff29a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:41.308627 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bb1491-b379-4935-a7e6-897752fd2a16 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:41.310762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 928/1857] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:12:41.314615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33859d06-dfb7-453d-9b09-1c0439eff29a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:41.316048 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8923af0-08be-47db-8bfd-b9abc826f2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:41.316650 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8923af0-08be-47db-8bfd-b9abc826f2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.316931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8923af0-08be-47db-8bfd-b9abc826f2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.317218 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8923af0-08be-47db-8bfd-b9abc826f2c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.317853 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8923af0-08be-47db-8bfd-b9abc826f2c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:41.319415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8923af0-08be-47db-8bfd-b9abc826f2c4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:41.324070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33859d06-dfb7-453d-9b09-1c0439eff29a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:41.328600 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b00e2d-772b-4bcb-9883-fb6cf9467554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:41.329258 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b00e2d-772b-4bcb-9883-fb6cf9467554 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.329402 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 930/1858] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:12:41.333260 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b00e2d-772b-4bcb-9883-fb6cf9467554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.333260 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b00e2d-772b-4bcb-9883-fb6cf9467554 admin admin] PATH_INFO: `/v3/projects/c84732a51f0741d6bdd015cff5aa5407/users/920b780ce85549ecae3d4a575ef52a35/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.333793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8923af0-08be-47db-8bfd-b9abc826f2c4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:41.344187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8923af0-08be-47db-8bfd-b9abc826f2c4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:41.345752 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b00e2d-772b-4bcb-9883-fb6cf9467554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c84732a51f0741d6bdd015cff5aa5407, user_id=920b780ce85549ecae3d4a575ef52a35, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:41.348386 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b00e2d-772b-4bcb-9883-fb6cf9467554 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:41.348785 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c849a2-f760-444b-86b6-e773c10c9436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:41.349592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c849a2-f760-444b-86b6-e773c10c9436 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.349875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c849a2-f760-444b-86b6-e773c10c9436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.350173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c849a2-f760-444b-86b6-e773c10c9436 admin admin] PATH_INFO: `/v3/projects/525224b48ff8419caad88dad16a5a9b2/users/bd5e67491e07475096a81355e2b2a4d8/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.352629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c849a2-f760-444b-86b6-e773c10c9436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=525224b48ff8419caad88dad16a5a9b2, user_id=bd5e67491e07475096a81355e2b2a4d8, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:41.355931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c849a2-f760-444b-86b6-e773c10c9436 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:41.362229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 929/1859] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:41 2022] PUT /identity/v3/projects/c84732a51f0741d6bdd015cff5aa5407/users/920b780ce85549ecae3d4a575ef52a35/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:12:41.369993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 931/1860] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:41 2022] PUT /identity/v3/projects/525224b48ff8419caad88dad16a5a9b2/users/bd5e67491e07475096a81355e2b2a4d8/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:12:41.373235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948c9f9-723a-42be-ace3-f99525df6b7a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.373525 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948c9f9-723a-42be-ace3-f99525df6b7a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.373790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6948c9f9-723a-42be-ace3-f99525df6b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.380955 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce09e7-3adc-46b6-8172-f9cded131e3d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.381267 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce09e7-3adc-46b6-8172-f9cded131e3d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.382498 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ce09e7-3adc-46b6-8172-f9cded131e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.401240 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6948c9f9-723a-42be-ace3-f99525df6b7a None None] MFA Rules not processed for user `920b780ce85549ecae3d4a575ef52a35`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:41.402348 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6948c9f9-723a-42be-ace3-f99525df6b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:41.407834 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:41.411098 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a9ce09e7-3adc-46b6-8172-f9cded131e3d None None] MFA Rules not processed for user `bd5e67491e07475096a81355e2b2a4d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:41.413591 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a9ce09e7-3adc-46b6-8172-f9cded131e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:41.444455 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:41.480024 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 930/1861] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:41.484753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948c9f9-723a-42be-ace3-f99525df6b7a None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:41.496340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 932/1862] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:41 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:12:41.508539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce09e7-3adc-46b6-8172-f9cded131e3d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:41.510158 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a9ce09e7-3adc-46b6-8172-f9cded131e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:41.554640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6948c9f9-723a-42be-ace3-f99525df6b7a None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:41.559089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841946e-9c56-45c9-acee-cfb87037f7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:41.559689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841946e-9c56-45c9-acee-cfb87037f7a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.559968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841946e-9c56-45c9-acee-cfb87037f7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.560232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4841946e-9c56-45c9-acee-cfb87037f7a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.560854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841946e-9c56-45c9-acee-cfb87037f7a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:41.564283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4841946e-9c56-45c9-acee-cfb87037f7a3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:41.584110 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ce09e7-3adc-46b6-8172-f9cded131e3d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:41.590577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ebd179-7c9a-4114-a195-28ab2207b201 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:41.597675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebd179-7c9a-4114-a195-28ab2207b201 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.598264 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebd179-7c9a-4114-a195-28ab2207b201 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.598759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ebd179-7c9a-4114-a195-28ab2207b201 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.604557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 931/1863] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:41 2022] POST /identity/v3/projects => generated 379 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:12:41.607181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841946e-9c56-45c9-acee-cfb87037f7a3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:41.611927 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ebd179-7c9a-4114-a195-28ab2207b201 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:41.625348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ebd179-7c9a-4114-a195-28ab2207b201 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:41.625465 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 933/1864] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:41 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:12:41.675655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4841946e-9c56-45c9-acee-cfb87037f7a3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:41.680249 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8eecf-bc93-4147-8f09-d9e55f327b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:41.680857 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8eecf-bc93-4147-8f09-d9e55f327b03 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.681796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8eecf-bc93-4147-8f09-d9e55f327b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.682072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f8eecf-bc93-4147-8f09-d9e55f327b03 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.682701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8eecf-bc93-4147-8f09-d9e55f327b03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:41.684717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f8eecf-bc93-4147-8f09-d9e55f327b03 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:41.706145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 932/1865] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:41 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:41.709980 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ebd179-7c9a-4114-a195-28ab2207b201 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:41.721555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ebd179-7c9a-4114-a195-28ab2207b201 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:41.726136 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5aaef-6b9b-4f15-ba78-6ae185300f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:41.726727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5aaef-6b9b-4f15-ba78-6ae185300f22 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.727218 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5aaef-6b9b-4f15-ba78-6ae185300f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.727583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd5aaef-6b9b-4f15-ba78-6ae185300f22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.728852 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd5aaef-6b9b-4f15-ba78-6ae185300f22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:41.730659 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd5aaef-6b9b-4f15-ba78-6ae185300f22 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:41.744412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 934/1866] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:41.746283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8eecf-bc93-4147-8f09-d9e55f327b03 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:41.756394 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f8eecf-bc93-4147-8f09-d9e55f327b03 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:41.761024 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167099c1-636a-4f04-ae37-171da5915340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:41.761658 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167099c1-636a-4f04-ae37-171da5915340 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.761927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167099c1-636a-4f04-ae37-171da5915340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.762193 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167099c1-636a-4f04-ae37-171da5915340 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.762827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167099c1-636a-4f04-ae37-171da5915340 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:41.764389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167099c1-636a-4f04-ae37-171da5915340 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:41.775126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 933/1867] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:41.779205 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5aaef-6b9b-4f15-ba78-6ae185300f22 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:41.789338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd5aaef-6b9b-4f15-ba78-6ae185300f22 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:41.795533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d93640-5885-4512-8de3-8df8bfe9cfcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:41.796188 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d93640-5885-4512-8de3-8df8bfe9cfcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.796480 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d93640-5885-4512-8de3-8df8bfe9cfcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.796733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d93640-5885-4512-8de3-8df8bfe9cfcd admin admin] PATH_INFO: `/v3/projects/06ba04cef35f4fae91841d9d5a8ac00b/users/36d2772af64745ebbd92bfebf55b3a89/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.812019 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d93640-5885-4512-8de3-8df8bfe9cfcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=06ba04cef35f4fae91841d9d5a8ac00b, user_id=36d2772af64745ebbd92bfebf55b3a89, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:41.814660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d93640-5885-4512-8de3-8df8bfe9cfcd admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:41.829395 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 935/1868] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:41 2022] PUT /identity/v3/projects/06ba04cef35f4fae91841d9d5a8ac00b/users/36d2772af64745ebbd92bfebf55b3a89/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:41.839761 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570af05-27ed-41f3-bd49-9602d122cea2 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:41.840175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570af05-27ed-41f3-bd49-9602d122cea2 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:41.840511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c570af05-27ed-41f3-bd49-9602d122cea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:41.872151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c570af05-27ed-41f3-bd49-9602d122cea2 None None] MFA Rules not processed for user `36d2772af64745ebbd92bfebf55b3a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:41.873795 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c570af05-27ed-41f3-bd49-9602d122cea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:41.904494 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:41.942588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 934/1869] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:41.947177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d93640-5885-4512-8de3-8df8bfe9cfcd admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:41.974766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570af05-27ed-41f3-bd49-9602d122cea2 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:42.011638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d93640-5885-4512-8de3-8df8bfe9cfcd admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:42.016311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b089a95-c76c-48de-813f-108bd6de094c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:42.016914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b089a95-c76c-48de-813f-108bd6de094c admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:42.017233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b089a95-c76c-48de-813f-108bd6de094c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:42.017494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b089a95-c76c-48de-813f-108bd6de094c admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:42.018120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b089a95-c76c-48de-813f-108bd6de094c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:42.020008 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b089a95-c76c-48de-813f-108bd6de094c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:42.031678 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:42.044823 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c570af05-27ed-41f3-bd49-9602d122cea2 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:42.049196 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 936/1870] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:41 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:42.049476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb36a5-b669-44fa-98e1-58ff5c0c5edf tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd5e67491e07475096a81355e2b2a4d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525224b48ff8419caad88dad16a5a9b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:42.050092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb36a5-b669-44fa-98e1-58ff5c0c5edf tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:42.050363 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb36a5-b669-44fa-98e1-58ff5c0c5edf tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:42.050641 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adb36a5-b669-44fa-98e1-58ff5c0c5edf tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:42.051284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb36a5-b669-44fa-98e1-58ff5c0c5edf tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:42.051589 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb36a5-b669-44fa-98e1-58ff5c0c5edf tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:42.053134 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b089a95-c76c-48de-813f-108bd6de094c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:42.053266 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adb36a5-b669-44fa-98e1-58ff5c0c5edf tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:42.065241 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 935/1871] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:12:42.069351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb36a5-b669-44fa-98e1-58ff5c0c5edf tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:42.078780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adb36a5-b669-44fa-98e1-58ff5c0c5edf tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:42.119404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b089a95-c76c-48de-813f-108bd6de094c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:42.124574 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db9dff-c2d6-4b99-b4d7-1e3453d83635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:42.125463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db9dff-c2d6-4b99-b4d7-1e3453d83635 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:42.125750 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db9dff-c2d6-4b99-b4d7-1e3453d83635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:42.126027 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63db9dff-c2d6-4b99-b4d7-1e3453d83635 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:42.126658 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db9dff-c2d6-4b99-b4d7-1e3453d83635 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:42.128599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63db9dff-c2d6-4b99-b4d7-1e3453d83635 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:42.142028 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c67f3d-2a26-45ec-b843-24851128c643 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd5e67491e07475096a81355e2b2a4d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525224b48ff8419caad88dad16a5a9b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:42.143444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c67f3d-2a26-45ec-b843-24851128c643 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:42.143444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c67f3d-2a26-45ec-b843-24851128c643 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:42.143648 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c67f3d-2a26-45ec-b843-24851128c643 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:42.144335 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c67f3d-2a26-45ec-b843-24851128c643 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:42.146512 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c67f3d-2a26-45ec-b843-24851128c643 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:42.153232 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 937/1872] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:42 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:42.157219 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db9dff-c2d6-4b99-b4d7-1e3453d83635 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:42.167572 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63db9dff-c2d6-4b99-b4d7-1e3453d83635 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:42.173764 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fbbc5-f290-4d69-b62f-ef184b1f83a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:42.174597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fbbc5-f290-4d69-b62f-ef184b1f83a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:42.174717 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fbbc5-f290-4d69-b62f-ef184b1f83a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:42.174958 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19fbbc5-f290-4d69-b62f-ef184b1f83a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:42.175662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19fbbc5-f290-4d69-b62f-ef184b1f83a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:42.177219 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19fbbc5-f290-4d69-b62f-ef184b1f83a5 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:42.179587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 936/1873] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:42 2022] POST /identity/v3/projects => generated 354 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:12:42.183479 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c67f3d-2a26-45ec-b843-24851128c643 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:42.193459 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 938/1874] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:12:42.195295 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fbbc5-f290-4d69-b62f-ef184b1f83a5 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:42.208079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19fbbc5-f290-4d69-b62f-ef184b1f83a5 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:42.260440 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:42.285475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c67f3d-2a26-45ec-b843-24851128c643 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:42.300287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6af4c0-44c7-4230-8a90-a9d13d7d4426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:42.301181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6af4c0-44c7-4230-8a90-a9d13d7d4426 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:42.301613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6af4c0-44c7-4230-8a90-a9d13d7d4426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:42.302083 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6af4c0-44c7-4230-8a90-a9d13d7d4426 admin admin] PATH_INFO: `/v3/projects/568db71cc131464eb7b7060444ffe2e7/users/7451c691e50a4d4ab4c33fcb7492f2b9/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:42.303943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d2bb4-4916-4d05-9037-8aa96f4ba898 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd5e67491e07475096a81355e2b2a4d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525224b48ff8419caad88dad16a5a9b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:42.304730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d2bb4-4916-4d05-9037-8aa96f4ba898 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:42.305017 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d2bb4-4916-4d05-9037-8aa96f4ba898 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:42.305317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26d2bb4-4916-4d05-9037-8aa96f4ba898 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:42.306017 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d2bb4-4916-4d05-9037-8aa96f4ba898 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:42.306414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d2bb4-4916-4d05-9037-8aa96f4ba898 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:42.308127 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26d2bb4-4916-4d05-9037-8aa96f4ba898 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:42.326677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6af4c0-44c7-4230-8a90-a9d13d7d4426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=568db71cc131464eb7b7060444ffe2e7, user_id=7451c691e50a4d4ab4c33fcb7492f2b9, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:42.329564 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6af4c0-44c7-4230-8a90-a9d13d7d4426 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:42.330504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 937/1875] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:12:42.364290 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 939/1876] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:42 2022] PUT /identity/v3/projects/568db71cc131464eb7b7060444ffe2e7/users/7451c691e50a4d4ab4c33fcb7492f2b9/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:42.368378 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d2bb4-4916-4d05-9037-8aa96f4ba898 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:42.379819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6af4c0-44c7-4230-8a90-a9d13d7d4426 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:42.383385 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26d2bb4-4916-4d05-9037-8aa96f4ba898 tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:42.438266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:42.453260 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6af4c0-44c7-4230-8a90-a9d13d7d4426 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:42.459388 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da699c8-3db4-493f-bc45-9251304a4cee tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bd5e67491e07475096a81355e2b2a4d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '525224b48ff8419caad88dad16a5a9b2', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:42.460315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff177061-5bf8-4717-8258-32ecfb6ba657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:42.460454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da699c8-3db4-493f-bc45-9251304a4cee tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:42.460454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da699c8-3db4-493f-bc45-9251304a4cee tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:42.460599 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff177061-5bf8-4717-8258-32ecfb6ba657 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:42.460681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff177061-5bf8-4717-8258-32ecfb6ba657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:42.460880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da699c8-3db4-493f-bc45-9251304a4cee tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] PATH_INFO: `/v3/projects/d1b63b591b834fb99849a2b4e31b552c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:42.460974 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff177061-5bf8-4717-8258-32ecfb6ba657 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:42.461773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff177061-5bf8-4717-8258-32ecfb6ba657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:42.462458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da699c8-3db4-493f-bc45-9251304a4cee tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] RBAC: Authorizing `identity:delete_project(project_id=d1b63b591b834fb99849a2b4e31b552c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:42.463752 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff177061-5bf8-4717-8258-32ecfb6ba657 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:42.465258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da699c8-3db4-493f-bc45-9251304a4cee tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:42.481171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 938/1877] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:42.485519 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff177061-5bf8-4717-8258-32ecfb6ba657 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:42.496770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff177061-5bf8-4717-8258-32ecfb6ba657 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:42.506351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859637fd-08c5-4a14-864d-a34670b17c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:42.507010 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859637fd-08c5-4a14-864d-a34670b17c16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:42.507292 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859637fd-08c5-4a14-864d-a34670b17c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:42.507564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859637fd-08c5-4a14-864d-a34670b17c16 admin admin] PATH_INFO: `/v3/projects/568db71cc131464eb7b7060444ffe2e7/users/7451c691e50a4d4ab4c33fcb7492f2b9/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:42.510056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859637fd-08c5-4a14-864d-a34670b17c16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=568db71cc131464eb7b7060444ffe2e7, user_id=7451c691e50a4d4ab4c33fcb7492f2b9, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:42.512721 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859637fd-08c5-4a14-864d-a34670b17c16 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:42.521579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2da699c8-3db4-493f-bc45-9251304a4cee tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1b63b591b834fb99849a2b4e31b552c', 'request_id': 'req-2da699c8-3db4-493f-bc45-9251304a4cee'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:42.524266 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 939/1878] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:42 2022] PUT /identity/v3/projects/568db71cc131464eb7b7060444ffe2e7/users/7451c691e50a4d4ab4c33fcb7492f2b9/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:12:42.529481 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2da699c8-3db4-493f-bc45-9251304a4cee tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1b63b591b834fb99849a2b4e31b552c', 'request_id': 'req-2da699c8-3db4-493f-bc45-9251304a4cee'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:42.534847 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc6f13-bb26-4f91-b335-0ea7432a117b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:42.535131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc6f13-bb26-4f91-b335-0ea7432a117b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:42.535402 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc6f13-bb26-4f91-b335-0ea7432a117b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:42.535505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2da699c8-3db4-493f-bc45-9251304a4cee tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1b63b591b834fb99849a2b4e31b552c', 'request_id': 'req-2da699c8-3db4-493f-bc45-9251304a4cee'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:42.547342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2da699c8-3db4-493f-bc45-9251304a4cee tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] The token cache is being invalidate because project d1b63b591b834fb99849a2b4e31b552c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:42.547753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2da699c8-3db4-493f-bc45-9251304a4cee tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:42.548486 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2da699c8-3db4-493f-bc45-9251304a4cee tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:42.550109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 940/1879] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:42 2022] DELETE /identity/v3/projects/d1b63b591b834fb99849a2b4e31b552c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:42.568759 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-01fc6f13-bb26-4f91-b335-0ea7432a117b None None] MFA Rules not processed for user `7451c691e50a4d4ab4c33fcb7492f2b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:42.570008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-01fc6f13-bb26-4f91-b335-0ea7432a117b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:42.653668 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 940/1880] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:12:42.665976 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da699c8-3db4-493f-bc45-9251304a4cee tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:42.667753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2da699c8-3db4-493f-bc45-9251304a4cee tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:42.739454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da699c8-3db4-493f-bc45-9251304a4cee tempest-VolumeQuotaClassesTest-599384689 tempest-VolumeQuotaClassesTest-599384689-project-admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:42.798373 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59000cf-1eca-44af-a6de-25a82a6778f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:42.799520 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59000cf-1eca-44af-a6de-25a82a6778f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:42.799975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59000cf-1eca-44af-a6de-25a82a6778f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:42.800417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59000cf-1eca-44af-a6de-25a82a6778f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:42.800678 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc6f13-bb26-4f91-b335-0ea7432a117b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:42.802164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-01fc6f13-bb26-4f91-b335-0ea7432a117b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:42.812916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59000cf-1eca-44af-a6de-25a82a6778f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:42.815487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59000cf-1eca-44af-a6de-25a82a6778f8 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:42.834758 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 941/1881] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:42.863298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:42.876734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59000cf-1eca-44af-a6de-25a82a6778f8 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:42.878131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b59000cf-1eca-44af-a6de-25a82a6778f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:42.879384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc6f13-bb26-4f91-b335-0ea7432a117b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:42.884890 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cb693e-694b-4488-812a-b30b58490787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:42.885556 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb693e-694b-4488-812a-b30b58490787 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:42.885827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb693e-694b-4488-812a-b30b58490787 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:42.886093 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cb693e-694b-4488-812a-b30b58490787 admin admin] PATH_INFO: `/v3/users/de13e3506df84492af284766b061ed4e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:42.887256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb693e-694b-4488-812a-b30b58490787 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de13e3506df84492af284766b061ed4e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:42.889166 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cb693e-694b-4488-812a-b30b58490787 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:42.889775 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59000cf-1eca-44af-a6de-25a82a6778f8 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:42.893476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce562a-1a2f-4783-be1c-f2abe0a6495d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:42.894070 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce562a-1a2f-4783-be1c-f2abe0a6495d service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:42.894351 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce562a-1a2f-4783-be1c-f2abe0a6495d service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:42.894633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce562a-1a2f-4783-be1c-f2abe0a6495d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:42.896328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-95ce562a-1a2f-4783-be1c-f2abe0a6495d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:42.936992 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-37cb693e-694b-4488-812a-b30b58490787 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de13e3506df84492af284766b061ed4e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:42.939817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:42.943991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-37cb693e-694b-4488-812a-b30b58490787 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de13e3506df84492af284766b061ed4e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:42.956973 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-37cb693e-694b-4488-812a-b30b58490787 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de13e3506df84492af284766b061ed4e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:42.957682 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-37cb693e-694b-4488-812a-b30b58490787 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de13e3506df84492af284766b061ed4e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:42.958354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-37cb693e-694b-4488-812a-b30b58490787 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de13e3506df84492af284766b061ed4e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:42.960146 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce562a-1a2f-4783-be1c-f2abe0a6495d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:42.961612 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce562a-1a2f-4783-be1c-f2abe0a6495d service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:42.972400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 941/1882] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:42 2022] DELETE /identity/v3/users/de13e3506df84492af284766b061ed4e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:42.972647 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 942/1883] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:42 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:42.976740 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df76f2-59cc-4e18-8f22-341268527772 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:42.977042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df76f2-59cc-4e18-8f22-341268527772 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:42.977342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df76f2-59cc-4e18-8f22-341268527772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:42.997458 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-66df76f2-59cc-4e18-8f22-341268527772 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:42.998742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-66df76f2-59cc-4e18-8f22-341268527772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:43.061901 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 942/1884] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:12:43.075506 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce562a-1a2f-4783-be1c-f2abe0a6495d service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:43.077017 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-95ce562a-1a2f-4783-be1c-f2abe0a6495d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:43.136797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce562a-1a2f-4783-be1c-f2abe0a6495d service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:43.189978 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddd750-d6fa-4e2e-a63e-62999ef13235 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:43.190583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddd750-d6fa-4e2e-a63e-62999ef13235 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:43.190860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddd750-d6fa-4e2e-a63e-62999ef13235 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:43.191128 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ddd750-d6fa-4e2e-a63e-62999ef13235 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:43.202155 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddd750-d6fa-4e2e-a63e-62999ef13235 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:43.203568 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ddd750-d6fa-4e2e-a63e-62999ef13235 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:43.214028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 943/1885] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:43.600369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df76f2-59cc-4e18-8f22-341268527772 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:43.601925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-66df76f2-59cc-4e18-8f22-341268527772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:43.616602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66df76f2-59cc-4e18-8f22-341268527772 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:43.621724 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f9bd0-7242-43d3-a647-2f0597284890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:43.622723 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f9bd0-7242-43d3-a647-2f0597284890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:43.623151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f9bd0-7242-43d3-a647-2f0597284890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:43.623526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905f9bd0-7242-43d3-a647-2f0597284890 admin admin] PATH_INFO: `/v3/projects/86c9d54d1be7403aa58dcd72c9fd41af` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:43.624829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f9bd0-7242-43d3-a647-2f0597284890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86c9d54d1be7403aa58dcd72c9fd41af)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:43.627384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905f9bd0-7242-43d3-a647-2f0597284890 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:43.679913 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-905f9bd0-7242-43d3-a647-2f0597284890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86c9d54d1be7403aa58dcd72c9fd41af', 'request_id': 'req-905f9bd0-7242-43d3-a647-2f0597284890'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:43.689565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-905f9bd0-7242-43d3-a647-2f0597284890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86c9d54d1be7403aa58dcd72c9fd41af', 'request_id': 'req-905f9bd0-7242-43d3-a647-2f0597284890'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:43.696422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-905f9bd0-7242-43d3-a647-2f0597284890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86c9d54d1be7403aa58dcd72c9fd41af', 'request_id': 'req-905f9bd0-7242-43d3-a647-2f0597284890'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:43.697819 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:43.711694 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-905f9bd0-7242-43d3-a647-2f0597284890 admin admin] The token cache is being invalidate because project 86c9d54d1be7403aa58dcd72c9fd41af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:43.713766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-905f9bd0-7242-43d3-a647-2f0597284890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:43.715290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-905f9bd0-7242-43d3-a647-2f0597284890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:43.716844 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 943/1886] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:43 2022] DELETE /identity/v3/projects/86c9d54d1be7403aa58dcd72c9fd41af => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:43.721482 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddd750-d6fa-4e2e-a63e-62999ef13235 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:43.733490 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ddd750-d6fa-4e2e-a63e-62999ef13235 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:43.793717 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dbb1c3-2ef7-4fe3-8256-448dadb41b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:43.794484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbb1c3-2ef7-4fe3-8256-448dadb41b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:43.794883 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbb1c3-2ef7-4fe3-8256-448dadb41b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:43.795224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dbb1c3-2ef7-4fe3-8256-448dadb41b76 admin admin] PATH_INFO: `/v3/users/bd5e67491e07475096a81355e2b2a4d8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:43.796411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dbb1c3-2ef7-4fe3-8256-448dadb41b76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd5e67491e07475096a81355e2b2a4d8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:43.798411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dbb1c3-2ef7-4fe3-8256-448dadb41b76 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:43.847988 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:43.858321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e8dbb1c3-2ef7-4fe3-8256-448dadb41b76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd5e67491e07475096a81355e2b2a4d8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:43.867374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e8dbb1c3-2ef7-4fe3-8256-448dadb41b76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd5e67491e07475096a81355e2b2a4d8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:43.875327 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e8dbb1c3-2ef7-4fe3-8256-448dadb41b76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd5e67491e07475096a81355e2b2a4d8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:43.876146 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e8dbb1c3-2ef7-4fe3-8256-448dadb41b76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd5e67491e07475096a81355e2b2a4d8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:43.889541 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e8dbb1c3-2ef7-4fe3-8256-448dadb41b76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd5e67491e07475096a81355e2b2a4d8'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:43.895866 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 944/1887] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:43 2022] DELETE /identity/v3/users/bd5e67491e07475096a81355e2b2a4d8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:44.194778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f9bd0-7242-43d3-a647-2f0597284890 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:44.198886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-905f9bd0-7242-43d3-a647-2f0597284890 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:44.229585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905f9bd0-7242-43d3-a647-2f0597284890 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:44.301814 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d80ade-79af-41e9-81d2-e3e445fa4368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:44.303237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d80ade-79af-41e9-81d2-e3e445fa4368 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:44.303237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d80ade-79af-41e9-81d2-e3e445fa4368 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:44.303237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d80ade-79af-41e9-81d2-e3e445fa4368 admin admin] PATH_INFO: `/v3/projects/525224b48ff8419caad88dad16a5a9b2` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:44.304131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d80ade-79af-41e9-81d2-e3e445fa4368 admin admin] RBAC: Authorizing `identity:delete_project(project_id=525224b48ff8419caad88dad16a5a9b2)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:44.306156 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d80ade-79af-41e9-81d2-e3e445fa4368 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:44.396695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-53d80ade-79af-41e9-81d2-e3e445fa4368 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '525224b48ff8419caad88dad16a5a9b2', 'request_id': 'req-53d80ade-79af-41e9-81d2-e3e445fa4368'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:44.423314 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-53d80ade-79af-41e9-81d2-e3e445fa4368 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '525224b48ff8419caad88dad16a5a9b2', 'request_id': 'req-53d80ade-79af-41e9-81d2-e3e445fa4368'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:44.432782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-53d80ade-79af-41e9-81d2-e3e445fa4368 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '525224b48ff8419caad88dad16a5a9b2', 'request_id': 'req-53d80ade-79af-41e9-81d2-e3e445fa4368'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:44.446165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-53d80ade-79af-41e9-81d2-e3e445fa4368 admin admin] The token cache is being invalidate because project 525224b48ff8419caad88dad16a5a9b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:44.446486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-53d80ade-79af-41e9-81d2-e3e445fa4368 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:44.447147 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-53d80ade-79af-41e9-81d2-e3e445fa4368 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:44.449245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 944/1888] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:44 2022] DELETE /identity/v3/projects/525224b48ff8419caad88dad16a5a9b2 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:44.454312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e4276-f305-4a6e-9c2b-0df9889e5e8a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:44.454623 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e4276-f305-4a6e-9c2b-0df9889e5e8a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:44.454871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68e4276-f305-4a6e-9c2b-0df9889e5e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:44.476836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-d68e4276-f305-4a6e-9c2b-0df9889e5e8a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:44.478636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d68e4276-f305-4a6e-9c2b-0df9889e5e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:44.547013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 945/1889] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:44.554787 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894de5c-1205-458c-8d42-98a2a2f14ff6 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:44.555075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894de5c-1205-458c-8d42-98a2a2f14ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:44.555349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894de5c-1205-458c-8d42-98a2a2f14ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:44.579165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b894de5c-1205-458c-8d42-98a2a2f14ff6 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:44.580696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b894de5c-1205-458c-8d42-98a2a2f14ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:44.591049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 945/1890] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:44.595552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e4276-f305-4a6e-9c2b-0df9889e5e8a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:44.626064 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68e4276-f305-4a6e-9c2b-0df9889e5e8a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:44.639586 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601c04d-a1ce-41b8-84aa-6ab7fbda5b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:44.641438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601c04d-a1ce-41b8-84aa-6ab7fbda5b08 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:44.643408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601c04d-a1ce-41b8-84aa-6ab7fbda5b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:44.643834 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601c04d-a1ce-41b8-84aa-6ab7fbda5b08 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:44.644698 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601c04d-a1ce-41b8-84aa-6ab7fbda5b08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:44.645554 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601c04d-a1ce-41b8-84aa-6ab7fbda5b08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:44.650474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601c04d-a1ce-41b8-84aa-6ab7fbda5b08 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:44.685570 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 946/1891] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:12:44.690290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894de5c-1205-458c-8d42-98a2a2f14ff6 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:44.703746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894de5c-1205-458c-8d42-98a2a2f14ff6 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:44.707776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9bc59-5f96-472e-b482-acd44134ad33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:44.708514 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9bc59-5f96-472e-b482-acd44134ad33 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:44.708887 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9bc59-5f96-472e-b482-acd44134ad33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:44.709287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe9bc59-5f96-472e-b482-acd44134ad33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:44.710018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9bc59-5f96-472e-b482-acd44134ad33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:44.712274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe9bc59-5f96-472e-b482-acd44134ad33 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:44.768793 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 946/1892] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:44 2022] POST /identity/v3/projects => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:44.772873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601c04d-a1ce-41b8-84aa-6ab7fbda5b08 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:44.790672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601c04d-a1ce-41b8-84aa-6ab7fbda5b08 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:44.916766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8cf7c4-5db4-4c59-af6a-c15647651120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:44.917701 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cf7c4-5db4-4c59-af6a-c15647651120 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:44.918131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cf7c4-5db4-4c59-af6a-c15647651120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:44.918462 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8cf7c4-5db4-4c59-af6a-c15647651120 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:44.919156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8cf7c4-5db4-4c59-af6a-c15647651120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:44.921374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8cf7c4-5db4-4c59-af6a-c15647651120 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:44.971944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 947/1893] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:44 2022] POST /identity/v3/users => generated 291 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:44.972793 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9bc59-5f96-472e-b482-acd44134ad33 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:44.994908 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe9bc59-5f96-472e-b482-acd44134ad33 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:45.002468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00b73af-4ba6-49f7-89f5-e6f236faf723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:45.003996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b73af-4ba6-49f7-89f5-e6f236faf723 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:45.004492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b73af-4ba6-49f7-89f5-e6f236faf723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:45.005117 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00b73af-4ba6-49f7-89f5-e6f236faf723 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:45.006135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b73af-4ba6-49f7-89f5-e6f236faf723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:45.011531 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00b73af-4ba6-49f7-89f5-e6f236faf723 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:45.024826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb60154-ad35-4196-a9e9-311bd3e9fb5e None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:45.024826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb60154-ad35-4196-a9e9-311bd3e9fb5e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:45.025732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb60154-ad35-4196-a9e9-311bd3e9fb5e None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:45.027304 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 948/1894] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 19:12:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:12:45.029237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 947/1895] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:12:45.033429 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb60154-ad35-4196-a9e9-311bd3e9fb5e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:45.035329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf87be5-5d97-438d-b331-e6b158ed2764 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:45.035583 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf87be5-5d97-438d-b331-e6b158ed2764 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:45.035876 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf87be5-5d97-438d-b331-e6b158ed2764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:45.052624 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb60154-ad35-4196-a9e9-311bd3e9fb5e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:45.061862 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ebf87be5-5d97-438d-b331-e6b158ed2764 None None] MFA Rules not processed for user `673a72ac9a9847e3812007b66ac7a5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:45.062929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ebf87be5-5d97-438d-b331-e6b158ed2764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:45.082024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbfde8-7478-42eb-807a-23141bad0b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:45.087006 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbfde8-7478-42eb-807a-23141bad0b87 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:45.087436 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbfde8-7478-42eb-807a-23141bad0b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:45.088263 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffbfde8-7478-42eb-807a-23141bad0b87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:45.089011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffbfde8-7478-42eb-807a-23141bad0b87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:45.105848 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffbfde8-7478-42eb-807a-23141bad0b87 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:45.134776 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 949/1896] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:12:45.143864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbfde8-7478-42eb-807a-23141bad0b87 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:45.158887 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffbfde8-7478-42eb-807a-23141bad0b87 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:45.175597 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:45.179159 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51ca7b6-c24c-4b6e-9b7f-2b3ad597dc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:45.180052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ca7b6-c24c-4b6e-9b7f-2b3ad597dc72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:45.180340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ca7b6-c24c-4b6e-9b7f-2b3ad597dc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:45.180621 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51ca7b6-c24c-4b6e-9b7f-2b3ad597dc72 admin admin] PATH_INFO: `/v3/projects/e1229b4a7f4f45eeaea4596ebcdae41d/users/7f56cd23a1db4bb086d6f751a3f0bcde/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:45.220612 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ca7b6-c24c-4b6e-9b7f-2b3ad597dc72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1229b4a7f4f45eeaea4596ebcdae41d, user_id=7f56cd23a1db4bb086d6f751a3f0bcde, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:45.223308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51ca7b6-c24c-4b6e-9b7f-2b3ad597dc72 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:45.231172 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 948/1897] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 19:12:45 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:45.244694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:45.256155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf87be5-5d97-438d-b331-e6b158ed2764 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:45.258686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 950/1898] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:45 2022] PUT /identity/v3/projects/e1229b4a7f4f45eeaea4596ebcdae41d/users/7f56cd23a1db4bb086d6f751a3f0bcde/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:45.260298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ebf87be5-5d97-438d-b331-e6b158ed2764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:45.270710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee61840-ec4b-461a-8e94-942cf1a07a8b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:45.271536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee61840-ec4b-461a-8e94-942cf1a07a8b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:45.271903 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee61840-ec4b-461a-8e94-942cf1a07a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:45.309836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf87be5-5d97-438d-b331-e6b158ed2764 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:45.313938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-dee61840-ec4b-461a-8e94-942cf1a07a8b None None] MFA Rules not processed for user `7f56cd23a1db4bb086d6f751a3f0bcde`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:45.315404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-dee61840-ec4b-461a-8e94-942cf1a07a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:45.406045 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 951/1899] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:45 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:12:45.408619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987b918-bc36-4e23-a2ff-ba519b0eca5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:45.409361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987b918-bc36-4e23-a2ff-ba519b0eca5f service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:45.409816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987b918-bc36-4e23-a2ff-ba519b0eca5f service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:45.410095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987b918-bc36-4e23-a2ff-ba519b0eca5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:45.410818 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee61840-ec4b-461a-8e94-942cf1a07a8b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:45.426604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987b918-bc36-4e23-a2ff-ba519b0eca5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:45.428405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4987b918-bc36-4e23-a2ff-ba519b0eca5f service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:45.442410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 949/1900] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:45.511213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee61840-ec4b-461a-8e94-942cf1a07a8b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:45.515622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6683984-a73d-4299-8f9f-b1d99155ecaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:45.516241 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6683984-a73d-4299-8f9f-b1d99155ecaa admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:45.516523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6683984-a73d-4299-8f9f-b1d99155ecaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:45.516790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6683984-a73d-4299-8f9f-b1d99155ecaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:45.517543 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6683984-a73d-4299-8f9f-b1d99155ecaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:45.519529 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6683984-a73d-4299-8f9f-b1d99155ecaa admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:45.563077 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 952/1901] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:45 2022] POST /identity/v3/projects => generated 379 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:45.567253 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987b918-bc36-4e23-a2ff-ba519b0eca5f service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:45.579154 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4987b918-bc36-4e23-a2ff-ba519b0eca5f service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:45.653877 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b277a70-ef3a-45d4-a02c-49b68bf25be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:45.654555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b277a70-ef3a-45d4-a02c-49b68bf25be1 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:45.654834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b277a70-ef3a-45d4-a02c-49b68bf25be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:45.655105 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b277a70-ef3a-45d4-a02c-49b68bf25be1 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:45.655733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b277a70-ef3a-45d4-a02c-49b68bf25be1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:45.657682 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b277a70-ef3a-45d4-a02c-49b68bf25be1 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:45.681467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 950/1902] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:45 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:12:45.685801 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6683984-a73d-4299-8f9f-b1d99155ecaa admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:45.698508 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6683984-a73d-4299-8f9f-b1d99155ecaa admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:45.703058 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6da60-fcde-4183-95c1-d625b9f08531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:45.703668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6da60-fcde-4183-95c1-d625b9f08531 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:45.703972 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6da60-fcde-4183-95c1-d625b9f08531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:45.704263 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc6da60-fcde-4183-95c1-d625b9f08531 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:45.704895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6da60-fcde-4183-95c1-d625b9f08531 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:45.706530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc6da60-fcde-4183-95c1-d625b9f08531 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:45.719978 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 953/1903] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:45.721850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b277a70-ef3a-45d4-a02c-49b68bf25be1 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:45.732076 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b277a70-ef3a-45d4-a02c-49b68bf25be1 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:45.736868 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a95d54-31e3-4cb0-b00d-a23d0234f39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:45.737549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a95d54-31e3-4cb0-b00d-a23d0234f39e admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:45.737821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a95d54-31e3-4cb0-b00d-a23d0234f39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:45.738086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a95d54-31e3-4cb0-b00d-a23d0234f39e admin admin] PATH_INFO: `/v3/projects/ff0a2f92bf8e4eafacbc7a5db5a8fe3c/users/a3401a4719f84ac199e7c79e141f4dc5/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:45.754125 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a95d54-31e3-4cb0-b00d-a23d0234f39e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff0a2f92bf8e4eafacbc7a5db5a8fe3c, user_id=a3401a4719f84ac199e7c79e141f4dc5, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:45.757275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a95d54-31e3-4cb0-b00d-a23d0234f39e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:45.774189 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 951/1904] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:45 2022] PUT /identity/v3/projects/ff0a2f92bf8e4eafacbc7a5db5a8fe3c/users/a3401a4719f84ac199e7c79e141f4dc5/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:45.778755 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6da60-fcde-4183-95c1-d625b9f08531 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:45.789245 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc6da60-fcde-4183-95c1-d625b9f08531 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:45.852190 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ec806-772d-4718-9859-38c6a2ee953d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:45.853352 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ec806-772d-4718-9859-38c6a2ee953d admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:45.853748 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ec806-772d-4718-9859-38c6a2ee953d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:45.854025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ec806-772d-4718-9859-38c6a2ee953d admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:45.854727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ec806-772d-4718-9859-38c6a2ee953d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:45.856411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ec806-772d-4718-9859-38c6a2ee953d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:45.863636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:45.870669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 954/1905] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:45 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:45.874726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a95d54-31e3-4cb0-b00d-a23d0234f39e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:45.885341 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a95d54-31e3-4cb0-b00d-a23d0234f39e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:45.894286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e5481-d46a-4fbf-8bf8-c7edf4baca36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:45.895061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e5481-d46a-4fbf-8bf8-c7edf4baca36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:45.895452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e5481-d46a-4fbf-8bf8-c7edf4baca36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:45.895821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5e5481-d46a-4fbf-8bf8-c7edf4baca36 admin admin] PATH_INFO: `/v3/projects/ff0a2f92bf8e4eafacbc7a5db5a8fe3c/users/a3401a4719f84ac199e7c79e141f4dc5/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:45.897976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e5481-d46a-4fbf-8bf8-c7edf4baca36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff0a2f92bf8e4eafacbc7a5db5a8fe3c, user_id=a3401a4719f84ac199e7c79e141f4dc5, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:45.900838 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5e5481-d46a-4fbf-8bf8-c7edf4baca36 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:45.912252 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 952/1906] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:45 2022] PUT /identity/v3/projects/ff0a2f92bf8e4eafacbc7a5db5a8fe3c/users/a3401a4719f84ac199e7c79e141f4dc5/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:12:45.925716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e62567-f592-44c7-bd90-45b925e6b8f4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:45.926069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e62567-f592-44c7-bd90-45b925e6b8f4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:45.926363 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e62567-f592-44c7-bd90-45b925e6b8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:45.959135 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-70e62567-f592-44c7-bd90-45b925e6b8f4 None None] MFA Rules not processed for user `a3401a4719f84ac199e7c79e141f4dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:45.960517 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-70e62567-f592-44c7-bd90-45b925e6b8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:46.070397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 955/1907] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:45 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:46.079223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e5481-d46a-4fbf-8bf8-c7edf4baca36 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:46.081182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-eb5e5481-d46a-4fbf-8bf8-c7edf4baca36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:46.105041 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:46.162802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5e5481-d46a-4fbf-8bf8-c7edf4baca36 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:46.166992 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19a12c-6977-4999-bf40-846b9a247c0b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:46.167602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19a12c-6977-4999-bf40-846b9a247c0b service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:46.167873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19a12c-6977-4999-bf40-846b9a247c0b service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:46.168170 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19a12c-6977-4999-bf40-846b9a247c0b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:46.182801 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19a12c-6977-4999-bf40-846b9a247c0b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:46.184231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19a12c-6977-4999-bf40-846b9a247c0b service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:46.195240 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 953/1908] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:46 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:46.649466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e62567-f592-44c7-bd90-45b925e6b8f4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:46.711946 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e62567-f592-44c7-bd90-45b925e6b8f4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:46.717350 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d548b-3556-41b0-bc91-d57aff6896d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:46.718295 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d548b-3556-41b0-bc91-d57aff6896d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:46.718548 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d548b-3556-41b0-bc91-d57aff6896d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:46.719086 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8d548b-3556-41b0-bc91-d57aff6896d0 admin admin] PATH_INFO: `/v3/users/7f56cd23a1db4bb086d6f751a3f0bcde` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:46.720324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d548b-3556-41b0-bc91-d57aff6896d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f56cd23a1db4bb086d6f751a3f0bcde)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:46.722567 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8d548b-3556-41b0-bc91-d57aff6896d0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:46.776068 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0f8d548b-3556-41b0-bc91-d57aff6896d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f56cd23a1db4bb086d6f751a3f0bcde'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:46.783660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0f8d548b-3556-41b0-bc91-d57aff6896d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f56cd23a1db4bb086d6f751a3f0bcde'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:46.791099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0f8d548b-3556-41b0-bc91-d57aff6896d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f56cd23a1db4bb086d6f751a3f0bcde'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:46.791696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0f8d548b-3556-41b0-bc91-d57aff6896d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f56cd23a1db4bb086d6f751a3f0bcde'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:46.805201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0f8d548b-3556-41b0-bc91-d57aff6896d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f56cd23a1db4bb086d6f751a3f0bcde'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:46.811980 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 956/1909] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:46 2022] DELETE /identity/v3/users/7f56cd23a1db4bb086d6f751a3f0bcde => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:46.816506 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ea52e-2a2f-489c-8a26-f1ad5a030128 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:46.816984 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ea52e-2a2f-489c-8a26-f1ad5a030128 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:46.817091 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ea52e-2a2f-489c-8a26-f1ad5a030128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:46.835805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f49ea52e-2a2f-489c-8a26-f1ad5a030128 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:46.836984 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f49ea52e-2a2f-489c-8a26-f1ad5a030128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:46.893482 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 954/1910] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:46.903268 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d548b-3556-41b0-bc91-d57aff6896d0 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:46.904765 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0f8d548b-3556-41b0-bc91-d57aff6896d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:46.911280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:46.966333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8d548b-3556-41b0-bc91-d57aff6896d0 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:46.969952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f9597-a255-404d-a164-aed8c3955aa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:46.970555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f9597-a255-404d-a164-aed8c3955aa7 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:46.970835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f9597-a255-404d-a164-aed8c3955aa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:46.971105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522f9597-a255-404d-a164-aed8c3955aa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:46.979995 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f9597-a255-404d-a164-aed8c3955aa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:46.981439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522f9597-a255-404d-a164-aed8c3955aa7 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:46.991038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 957/1911] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:47.459689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ea52e-2a2f-489c-8a26-f1ad5a030128 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:47.461472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f49ea52e-2a2f-489c-8a26-f1ad5a030128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:47.475760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ea52e-2a2f-489c-8a26-f1ad5a030128 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:47.481590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c19c1-cdcc-44ac-8e97-f032c94c8555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:47.482359 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c19c1-cdcc-44ac-8e97-f032c94c8555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:47.482743 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c19c1-cdcc-44ac-8e97-f032c94c8555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:47.483142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097c19c1-cdcc-44ac-8e97-f032c94c8555 admin admin] PATH_INFO: `/v3/projects/e1229b4a7f4f45eeaea4596ebcdae41d` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:47.484701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c19c1-cdcc-44ac-8e97-f032c94c8555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1229b4a7f4f45eeaea4596ebcdae41d)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:47.486798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097c19c1-cdcc-44ac-8e97-f032c94c8555 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:47.594839 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-097c19c1-cdcc-44ac-8e97-f032c94c8555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1229b4a7f4f45eeaea4596ebcdae41d', 'request_id': 'req-097c19c1-cdcc-44ac-8e97-f032c94c8555'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:47.605211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-097c19c1-cdcc-44ac-8e97-f032c94c8555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1229b4a7f4f45eeaea4596ebcdae41d', 'request_id': 'req-097c19c1-cdcc-44ac-8e97-f032c94c8555'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:47.614375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-097c19c1-cdcc-44ac-8e97-f032c94c8555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1229b4a7f4f45eeaea4596ebcdae41d', 'request_id': 'req-097c19c1-cdcc-44ac-8e97-f032c94c8555'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:47.629863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-097c19c1-cdcc-44ac-8e97-f032c94c8555 admin admin] The token cache is being invalidate because project e1229b4a7f4f45eeaea4596ebcdae41d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:47.633099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-097c19c1-cdcc-44ac-8e97-f032c94c8555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:47.634013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-097c19c1-cdcc-44ac-8e97-f032c94c8555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:47.636576 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 955/1912] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:47 2022] DELETE /identity/v3/projects/e1229b4a7f4f45eeaea4596ebcdae41d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:47.640590 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f9597-a255-404d-a164-aed8c3955aa7 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:47.656518 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522f9597-a255-404d-a164-aed8c3955aa7 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:47.740314 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8fc74-20fe-4ebe-9f6a-d8e654441494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:47.741161 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8fc74-20fe-4ebe-9f6a-d8e654441494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:47.741594 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8fc74-20fe-4ebe-9f6a-d8e654441494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:47.742039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8fc74-20fe-4ebe-9f6a-d8e654441494 admin admin] PATH_INFO: `/v3/users/a3401a4719f84ac199e7c79e141f4dc5` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:47.743860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc8fc74-20fe-4ebe-9f6a-d8e654441494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3401a4719f84ac199e7c79e141f4dc5)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:47.746320 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc8fc74-20fe-4ebe-9f6a-d8e654441494 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:47.832367 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0bc8fc74-20fe-4ebe-9f6a-d8e654441494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3401a4719f84ac199e7c79e141f4dc5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:47.870448 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0bc8fc74-20fe-4ebe-9f6a-d8e654441494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3401a4719f84ac199e7c79e141f4dc5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:47.903612 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0bc8fc74-20fe-4ebe-9f6a-d8e654441494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3401a4719f84ac199e7c79e141f4dc5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:47.904585 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0bc8fc74-20fe-4ebe-9f6a-d8e654441494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3401a4719f84ac199e7c79e141f4dc5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:47.931647 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0bc8fc74-20fe-4ebe-9f6a-d8e654441494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3401a4719f84ac199e7c79e141f4dc5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:47.941733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 958/1913] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:47 2022] DELETE /identity/v3/users/a3401a4719f84ac199e7c79e141f4dc5 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:48.939816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c19c1-cdcc-44ac-8e97-f032c94c8555 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:48.941527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-097c19c1-cdcc-44ac-8e97-f032c94c8555 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:48.956258 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097c19c1-cdcc-44ac-8e97-f032c94c8555 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:49.023931 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:49.051351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e38be7-71bd-4134-8e31-fa77e6c60c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:49.052284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e38be7-71bd-4134-8e31-fa77e6c60c2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:49.052572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e38be7-71bd-4134-8e31-fa77e6c60c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:49.052852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e38be7-71bd-4134-8e31-fa77e6c60c2e admin admin] PATH_INFO: `/v3/projects/ff0a2f92bf8e4eafacbc7a5db5a8fe3c` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:49.054200 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e38be7-71bd-4134-8e31-fa77e6c60c2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff0a2f92bf8e4eafacbc7a5db5a8fe3c)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:49.057346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e38be7-71bd-4134-8e31-fa77e6c60c2e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:49.106783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e5e38be7-71bd-4134-8e31-fa77e6c60c2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff0a2f92bf8e4eafacbc7a5db5a8fe3c', 'request_id': 'req-e5e38be7-71bd-4134-8e31-fa77e6c60c2e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:49.116352 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e5e38be7-71bd-4134-8e31-fa77e6c60c2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff0a2f92bf8e4eafacbc7a5db5a8fe3c', 'request_id': 'req-e5e38be7-71bd-4134-8e31-fa77e6c60c2e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:49.123466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e5e38be7-71bd-4134-8e31-fa77e6c60c2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff0a2f92bf8e4eafacbc7a5db5a8fe3c', 'request_id': 'req-e5e38be7-71bd-4134-8e31-fa77e6c60c2e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:49.136192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e5e38be7-71bd-4134-8e31-fa77e6c60c2e admin admin] The token cache is being invalidate because project ff0a2f92bf8e4eafacbc7a5db5a8fe3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:49.136517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e5e38be7-71bd-4134-8e31-fa77e6c60c2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:49.137363 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e5e38be7-71bd-4134-8e31-fa77e6c60c2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:49.139056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 956/1914] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:48 2022] DELETE /identity/v3/projects/ff0a2f92bf8e4eafacbc7a5db5a8fe3c => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:49.145292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292e16b-4fb6-4da9-880d-2a480a39a2e6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:49.145795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292e16b-4fb6-4da9-880d-2a480a39a2e6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:49.145880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6292e16b-4fb6-4da9-880d-2a480a39a2e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:49.171777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-6292e16b-4fb6-4da9-880d-2a480a39a2e6 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:49.173454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6292e16b-4fb6-4da9-880d-2a480a39a2e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:49.200290 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:49.267464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 959/1915] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:49.275724 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31531a2-f6b8-4ea4-9603-3081017841f4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:49.276013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31531a2-f6b8-4ea4-9603-3081017841f4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:49.276288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31531a2-f6b8-4ea4-9603-3081017841f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:49.307809 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e31531a2-f6b8-4ea4-9603-3081017841f4 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:49.309811 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e31531a2-f6b8-4ea4-9603-3081017841f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:49.321764 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 957/1916] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:49.326139 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292e16b-4fb6-4da9-880d-2a480a39a2e6 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:49.336976 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6292e16b-4fb6-4da9-880d-2a480a39a2e6 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:49.344179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa2ccd-bc02-4337-a4db-00174736257f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:49.344977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa2ccd-bc02-4337-a4db-00174736257f admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:49.345538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa2ccd-bc02-4337-a4db-00174736257f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:49.345814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fa2ccd-bc02-4337-a4db-00174736257f admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:49.346679 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa2ccd-bc02-4337-a4db-00174736257f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:49.347209 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa2ccd-bc02-4337-a4db-00174736257f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:49.349532 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fa2ccd-bc02-4337-a4db-00174736257f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:49.373175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31531a2-f6b8-4ea4-9603-3081017841f4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:49.377109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 960/1917] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:12:49.378347 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa2ccd-bc02-4337-a4db-00174736257f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:49.378467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e31531a2-f6b8-4ea4-9603-3081017841f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:49.389282 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fa2ccd-bc02-4337-a4db-00174736257f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:49.390552 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31531a2-f6b8-4ea4-9603-3081017841f4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:49.394691 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805dacc6-9678-49c4-9b8b-4f2a12a640a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:49.395322 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805dacc6-9678-49c4-9b8b-4f2a12a640a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:49.395550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdec330-85f2-46d1-8b2e-f6a22a10cf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:49.395673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805dacc6-9678-49c4-9b8b-4f2a12a640a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:49.395859 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805dacc6-9678-49c4-9b8b-4f2a12a640a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:49.396570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdec330-85f2-46d1-8b2e-f6a22a10cf36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:49.396683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805dacc6-9678-49c4-9b8b-4f2a12a640a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:49.397082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdec330-85f2-46d1-8b2e-f6a22a10cf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:49.397532 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdec330-85f2-46d1-8b2e-f6a22a10cf36 admin admin] PATH_INFO: `/v3/users/920b780ce85549ecae3d4a575ef52a35` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:49.398506 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805dacc6-9678-49c4-9b8b-4f2a12a640a3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:49.399002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdec330-85f2-46d1-8b2e-f6a22a10cf36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=920b780ce85549ecae3d4a575ef52a35)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:49.401575 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdec330-85f2-46d1-8b2e-f6a22a10cf36 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:49.429218 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 961/1918] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:49 2022] POST /identity/v3/projects => generated 383 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:12:49.433545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805dacc6-9678-49c4-9b8b-4f2a12a640a3 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:49.450829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-acdec330-85f2-46d1-8b2e-f6a22a10cf36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920b780ce85549ecae3d4a575ef52a35'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:49.458122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-acdec330-85f2-46d1-8b2e-f6a22a10cf36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920b780ce85549ecae3d4a575ef52a35'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:49.470980 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-acdec330-85f2-46d1-8b2e-f6a22a10cf36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920b780ce85549ecae3d4a575ef52a35'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:49.471756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-acdec330-85f2-46d1-8b2e-f6a22a10cf36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920b780ce85549ecae3d4a575ef52a35'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:49.472456 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-acdec330-85f2-46d1-8b2e-f6a22a10cf36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920b780ce85549ecae3d4a575ef52a35'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:49.485924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 958/1919] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:49 2022] DELETE /identity/v3/users/920b780ce85549ecae3d4a575ef52a35 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:49.490262 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dc6a0-59ca-403c-8d20-3117493aa850 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:49.490571 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dc6a0-59ca-403c-8d20-3117493aa850 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:49.490841 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3dc6a0-59ca-403c-8d20-3117493aa850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:49.498970 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805dacc6-9678-49c4-9b8b-4f2a12a640a3 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:49.509994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9a3dc6a0-59ca-403c-8d20-3117493aa850 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:49.511189 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9a3dc6a0-59ca-403c-8d20-3117493aa850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:49.542029 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:49.557909 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cba70-90e0-49f3-aea6-97ea52bdcf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:49.558579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cba70-90e0-49f3-aea6-97ea52bdcf09 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:49.558901 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cba70-90e0-49f3-aea6-97ea52bdcf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:49.559172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cba70-90e0-49f3-aea6-97ea52bdcf09 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:49.559794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3cba70-90e0-49f3-aea6-97ea52bdcf09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:49.561727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3cba70-90e0-49f3-aea6-97ea52bdcf09 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:49.571378 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 959/1920] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:12:49.582337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dc6a0-59ca-403c-8d20-3117493aa850 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:49.583415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 962/1921] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:49 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:12:49.583947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9a3dc6a0-59ca-403c-8d20-3117493aa850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:49.587869 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cba70-90e0-49f3-aea6-97ea52bdcf09 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:49.589495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8f3cba70-90e0-49f3-aea6-97ea52bdcf09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:49.601049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cba70-90e0-49f3-aea6-97ea52bdcf09 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:49.606082 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7a958-ed57-4358-ac14-287acee933af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:49.606673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7a958-ed57-4358-ac14-287acee933af admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:49.606951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7a958-ed57-4358-ac14-287acee933af admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:49.607224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f7a958-ed57-4358-ac14-287acee933af admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:49.607857 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7a958-ed57-4358-ac14-287acee933af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:49.609725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f7a958-ed57-4358-ac14-287acee933af admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:49.623746 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 963/1922] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:12:49.625289 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7a958-ed57-4358-ac14-287acee933af admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:49.635435 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f7a958-ed57-4358-ac14-287acee933af admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:49.640010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1be86-16df-40cf-9b9d-e70770a9fbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:49.640597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1be86-16df-40cf-9b9d-e70770a9fbcc admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:49.640880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1be86-16df-40cf-9b9d-e70770a9fbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:49.641156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b1be86-16df-40cf-9b9d-e70770a9fbcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:49.641824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1be86-16df-40cf-9b9d-e70770a9fbcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:49.643397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b1be86-16df-40cf-9b9d-e70770a9fbcc admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:49.647618 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3dc6a0-59ca-403c-8d20-3117493aa850 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:49.651973 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf733ad-2605-41a9-8c42-4a3066d5e642 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:49.652583 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf733ad-2605-41a9-8c42-4a3066d5e642 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:49.652859 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf733ad-2605-41a9-8c42-4a3066d5e642 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:49.653130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf733ad-2605-41a9-8c42-4a3066d5e642 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:49.654383 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 964/1923] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:12:49.658765 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1be86-16df-40cf-9b9d-e70770a9fbcc admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:49.667804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf733ad-2605-41a9-8c42-4a3066d5e642 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:49.668468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b1be86-16df-40cf-9b9d-e70770a9fbcc admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:49.669353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf733ad-2605-41a9-8c42-4a3066d5e642 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:49.674687 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc159c-c1ad-48d5-ac18-83e97f9bde44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:49.675393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc159c-c1ad-48d5-ac18-83e97f9bde44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:49.675675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc159c-c1ad-48d5-ac18-83e97f9bde44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:49.675986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcc159c-c1ad-48d5-ac18-83e97f9bde44 admin admin] PATH_INFO: `/v3/projects/2250e2ff2c524c328865f1752dc98a6d/users/6abc049e0f024d559b2eb82ecaf45959/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:49.678443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:49.685607 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 960/1924] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:12:49.694631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc159c-c1ad-48d5-ac18-83e97f9bde44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2250e2ff2c524c328865f1752dc98a6d, user_id=6abc049e0f024d559b2eb82ecaf45959, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:49.698190 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcc159c-c1ad-48d5-ac18-83e97f9bde44 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:49.715308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 965/1925] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:49 2022] PUT /identity/v3/projects/2250e2ff2c524c328865f1752dc98a6d/users/6abc049e0f024d559b2eb82ecaf45959/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:49.726621 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c4133-0c24-4d0e-86c5-e6e7c9a28143 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:49.727134 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c4133-0c24-4d0e-86c5-e6e7c9a28143 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:49.727412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c4133-0c24-4d0e-86c5-e6e7c9a28143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:49.770236 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c47c4133-0c24-4d0e-86c5-e6e7c9a28143 None None] MFA Rules not processed for user `6abc049e0f024d559b2eb82ecaf45959`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:49.771444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c47c4133-0c24-4d0e-86c5-e6e7c9a28143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:49.852971 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc159c-c1ad-48d5-ac18-83e97f9bde44 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:49.853472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 961/1926] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:49.943103 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcc159c-c1ad-48d5-ac18-83e97f9bde44 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:49.950094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a783a-c5bb-4555-9783-0cd5ee54f8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:49.952049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a783a-c5bb-4555-9783-0cd5ee54f8ea admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:49.952337 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a783a-c5bb-4555-9783-0cd5ee54f8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:49.952610 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338a783a-c5bb-4555-9783-0cd5ee54f8ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:49.953278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a783a-c5bb-4555-9783-0cd5ee54f8ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:49.957232 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338a783a-c5bb-4555-9783-0cd5ee54f8ea admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:49.992774 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 966/1927] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:49 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:49.995068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c4133-0c24-4d0e-86c5-e6e7c9a28143 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:50.007412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c4133-0c24-4d0e-86c5-e6e7c9a28143 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:50.008648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a783a-c5bb-4555-9783-0cd5ee54f8ea admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:50.011088 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-338a783a-c5bb-4555-9783-0cd5ee54f8ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:50.101583 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad3a2a-dae2-496f-bec9-74f918192238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:50.103692 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad3a2a-dae2-496f-bec9-74f918192238 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:50.104176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad3a2a-dae2-496f-bec9-74f918192238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:50.104456 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ad3a2a-dae2-496f-bec9-74f918192238 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:50.106611 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad3a2a-dae2-496f-bec9-74f918192238 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:50.109936 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338a783a-c5bb-4555-9783-0cd5ee54f8ea admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:50.114188 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ad3a2a-dae2-496f-bec9-74f918192238 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:50.115827 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8f391-94e5-4bf8-b280-80500c215ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:50.117233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8f391-94e5-4bf8-b280-80500c215ff0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:50.117635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8f391-94e5-4bf8-b280-80500c215ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:50.118010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8f391-94e5-4bf8-b280-80500c215ff0 admin admin] PATH_INFO: `/v3/projects/c84732a51f0741d6bdd015cff5aa5407` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:50.119294 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8f391-94e5-4bf8-b280-80500c215ff0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c84732a51f0741d6bdd015cff5aa5407)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:50.121525 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8f391-94e5-4bf8-b280-80500c215ff0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:50.133429 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:50.148672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 962/1928] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:49 2022] POST /identity/v3/users => generated 299 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:50.151574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad3a2a-dae2-496f-bec9-74f918192238 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:50.165160 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ad3a2a-dae2-496f-bec9-74f918192238 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:50.171692 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad854add-4ac7-49dd-ba0d-6c00753595ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:50.172314 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad854add-4ac7-49dd-ba0d-6c00753595ae admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:50.172589 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad854add-4ac7-49dd-ba0d-6c00753595ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:50.172858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad854add-4ac7-49dd-ba0d-6c00753595ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:50.173542 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad854add-4ac7-49dd-ba0d-6c00753595ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:50.175155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad854add-4ac7-49dd-ba0d-6c00753595ae admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:50.189133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 963/1929] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:50.190310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad854add-4ac7-49dd-ba0d-6c00753595ae admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:50.193015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6cc8f391-94e5-4bf8-b280-80500c215ff0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c84732a51f0741d6bdd015cff5aa5407', 'request_id': 'req-6cc8f391-94e5-4bf8-b280-80500c215ff0'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:50.200458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6cc8f391-94e5-4bf8-b280-80500c215ff0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c84732a51f0741d6bdd015cff5aa5407', 'request_id': 'req-6cc8f391-94e5-4bf8-b280-80500c215ff0'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:50.204545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad854add-4ac7-49dd-ba0d-6c00753595ae admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:50.207082 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6cc8f391-94e5-4bf8-b280-80500c215ff0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c84732a51f0741d6bdd015cff5aa5407', 'request_id': 'req-6cc8f391-94e5-4bf8-b280-80500c215ff0'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:50.225549 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:50.231964 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6cc8f391-94e5-4bf8-b280-80500c215ff0 admin admin] The token cache is being invalidate because project c84732a51f0741d6bdd015cff5aa5407 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:50.232303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6cc8f391-94e5-4bf8-b280-80500c215ff0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:50.233796 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6cc8f391-94e5-4bf8-b280-80500c215ff0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:50.241146 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 967/1930] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:50 2022] DELETE /identity/v3/projects/c84732a51f0741d6bdd015cff5aa5407 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:50.242151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8f391-94e5-4bf8-b280-80500c215ff0 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:50.243677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6cc8f391-94e5-4bf8-b280-80500c215ff0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:50.282919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e589ba-c1e7-45a1-82f2-758af086d285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:50.284164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e589ba-c1e7-45a1-82f2-758af086d285 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:50.284902 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e589ba-c1e7-45a1-82f2-758af086d285 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:50.285301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e589ba-c1e7-45a1-82f2-758af086d285 admin admin] PATH_INFO: `/v3/projects/66319069c1e647cc9f193e9b6e8d10bd/users/7b427a3e4fa04eef9556ec83e76f8595/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:50.305607 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e589ba-c1e7-45a1-82f2-758af086d285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66319069c1e647cc9f193e9b6e8d10bd, user_id=7b427a3e4fa04eef9556ec83e76f8595, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:50.308527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e589ba-c1e7-45a1-82f2-758af086d285 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:50.321060 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8f391-94e5-4bf8-b280-80500c215ff0 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:50.326203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 964/1931] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:50 2022] PUT /identity/v3/projects/66319069c1e647cc9f193e9b6e8d10bd/users/7b427a3e4fa04eef9556ec83e76f8595/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:50.330860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e589ba-c1e7-45a1-82f2-758af086d285 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:50.333207 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d1e589ba-c1e7-45a1-82f2-758af086d285 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:50.432261 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e589ba-c1e7-45a1-82f2-758af086d285 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:50.439603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6268735-6db6-47ca-b91d-c35ea6e00457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:50.440272 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6268735-6db6-47ca-b91d-c35ea6e00457 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:50.440549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6268735-6db6-47ca-b91d-c35ea6e00457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:50.440809 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6268735-6db6-47ca-b91d-c35ea6e00457 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:50.441667 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6268735-6db6-47ca-b91d-c35ea6e00457 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:50.443555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6268735-6db6-47ca-b91d-c35ea6e00457 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:50.445270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b3fe1-aa51-4b98-b37b-7b84b45d0834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:50.446024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b3fe1-aa51-4b98-b37b-7b84b45d0834 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:50.446368 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b3fe1-aa51-4b98-b37b-7b84b45d0834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:50.448545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5b3fe1-aa51-4b98-b37b-7b84b45d0834 admin admin] PATH_INFO: `/v3/users/36d2772af64745ebbd92bfebf55b3a89` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:50.451401 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b3fe1-aa51-4b98-b37b-7b84b45d0834 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36d2772af64745ebbd92bfebf55b3a89)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:50.453749 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5b3fe1-aa51-4b98-b37b-7b84b45d0834 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:50.462790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 965/1932] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:50 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:50.464115 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6268735-6db6-47ca-b91d-c35ea6e00457 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:50.487030 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6268735-6db6-47ca-b91d-c35ea6e00457 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:50.496048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86613d41-4684-4b13-950b-abf5acd4bbeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:50.497393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86613d41-4684-4b13-950b-abf5acd4bbeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:50.497624 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86613d41-4684-4b13-950b-abf5acd4bbeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:50.497912 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86613d41-4684-4b13-950b-abf5acd4bbeb admin admin] PATH_INFO: `/v3/projects/66319069c1e647cc9f193e9b6e8d10bd/users/7b427a3e4fa04eef9556ec83e76f8595/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:50.501152 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86613d41-4684-4b13-950b-abf5acd4bbeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=66319069c1e647cc9f193e9b6e8d10bd, user_id=7b427a3e4fa04eef9556ec83e76f8595, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:50.504574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86613d41-4684-4b13-950b-abf5acd4bbeb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:50.517694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ab5b3fe1-aa51-4b98-b37b-7b84b45d0834 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36d2772af64745ebbd92bfebf55b3a89'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:50.522067 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 966/1933] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:50 2022] PUT /identity/v3/projects/66319069c1e647cc9f193e9b6e8d10bd/users/7b427a3e4fa04eef9556ec83e76f8595/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:12:50.526325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ab5b3fe1-aa51-4b98-b37b-7b84b45d0834 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36d2772af64745ebbd92bfebf55b3a89'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:50.528921 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69b84b-7145-4620-8421-01af52c836f5 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:50.529430 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69b84b-7145-4620-8421-01af52c836f5 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:50.529708 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f69b84b-7145-4620-8421-01af52c836f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:50.549390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ab5b3fe1-aa51-4b98-b37b-7b84b45d0834 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36d2772af64745ebbd92bfebf55b3a89'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:50.549390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ab5b3fe1-aa51-4b98-b37b-7b84b45d0834 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36d2772af64745ebbd92bfebf55b3a89'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:50.568052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ab5b3fe1-aa51-4b98-b37b-7b84b45d0834 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36d2772af64745ebbd92bfebf55b3a89'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:50.570477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0f69b84b-7145-4620-8421-01af52c836f5 None None] MFA Rules not processed for user `7b427a3e4fa04eef9556ec83e76f8595`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:50.571642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0f69b84b-7145-4620-8421-01af52c836f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:50.576342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 968/1934] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:50 2022] DELETE /identity/v3/users/36d2772af64745ebbd92bfebf55b3a89 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:50.622834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:50.654593 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 967/1935] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:12:50.666418 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b3fe1-aa51-4b98-b37b-7b84b45d0834 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:50.667864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ab5b3fe1-aa51-4b98-b37b-7b84b45d0834 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:50.699274 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:50.735341 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5b3fe1-aa51-4b98-b37b-7b84b45d0834 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:50.739003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71a6e7-de5c-43c4-a479-407e255aa8c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:50.739618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71a6e7-de5c-43c4-a479-407e255aa8c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:50.740044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71a6e7-de5c-43c4-a479-407e255aa8c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:50.740322 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c71a6e7-de5c-43c4-a479-407e255aa8c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:50.742963 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4c71a6e7-de5c-43c4-a479-407e255aa8c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:50.806887 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c71a6e7-de5c-43c4-a479-407e255aa8c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:50.808501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c71a6e7-de5c-43c4-a479-407e255aa8c2 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:50.817450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 969/1936] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:12:50.861884 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69b84b-7145-4620-8421-01af52c836f5 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:50.863883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0f69b84b-7145-4620-8421-01af52c836f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:50.939155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f69b84b-7145-4620-8421-01af52c836f5 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:51.000603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9fad8-88b1-463b-bd33-89cc51d257aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:51.001300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9fad8-88b1-463b-bd33-89cc51d257aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:51.001540 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9fad8-88b1-463b-bd33-89cc51d257aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:51.001807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9fad8-88b1-463b-bd33-89cc51d257aa admin admin] PATH_INFO: `/v3/projects/06ba04cef35f4fae91841d9d5a8ac00b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:51.002920 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9fad8-88b1-463b-bd33-89cc51d257aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=06ba04cef35f4fae91841d9d5a8ac00b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:51.004813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd9fad8-88b1-463b-bd33-89cc51d257aa admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:51.058071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0bd9fad8-88b1-463b-bd33-89cc51d257aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06ba04cef35f4fae91841d9d5a8ac00b', 'request_id': 'req-0bd9fad8-88b1-463b-bd33-89cc51d257aa'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:51.066099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0bd9fad8-88b1-463b-bd33-89cc51d257aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06ba04cef35f4fae91841d9d5a8ac00b', 'request_id': 'req-0bd9fad8-88b1-463b-bd33-89cc51d257aa'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:51.073758 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0bd9fad8-88b1-463b-bd33-89cc51d257aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06ba04cef35f4fae91841d9d5a8ac00b', 'request_id': 'req-0bd9fad8-88b1-463b-bd33-89cc51d257aa'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:51.089015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0bd9fad8-88b1-463b-bd33-89cc51d257aa admin admin] The token cache is being invalidate because project 06ba04cef35f4fae91841d9d5a8ac00b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:51.089343 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0bd9fad8-88b1-463b-bd33-89cc51d257aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:51.090182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0bd9fad8-88b1-463b-bd33-89cc51d257aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:51.092023 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 968/1937] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:50 2022] DELETE /identity/v3/projects/06ba04cef35f4fae91841d9d5a8ac00b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:51.095828 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71a6e7-de5c-43c4-a479-407e255aa8c2 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:51.107533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c71a6e7-de5c-43c4-a479-407e255aa8c2 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:51.186486 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2312da2-a683-4b72-a61e-693f667271c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:51.187111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2312da2-a683-4b72-a61e-693f667271c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:51.187392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2312da2-a683-4b72-a61e-693f667271c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:51.187724 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2312da2-a683-4b72-a61e-693f667271c3 admin admin] PATH_INFO: `/v3/users/7451c691e50a4d4ab4c33fcb7492f2b9` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:51.188890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2312da2-a683-4b72-a61e-693f667271c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7451c691e50a4d4ab4c33fcb7492f2b9)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:51.194878 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2312da2-a683-4b72-a61e-693f667271c3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:51.256758 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d2312da2-a683-4b72-a61e-693f667271c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7451c691e50a4d4ab4c33fcb7492f2b9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:51.264288 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d2312da2-a683-4b72-a61e-693f667271c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7451c691e50a4d4ab4c33fcb7492f2b9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:51.272248 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d2312da2-a683-4b72-a61e-693f667271c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7451c691e50a4d4ab4c33fcb7492f2b9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:51.272982 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d2312da2-a683-4b72-a61e-693f667271c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7451c691e50a4d4ab4c33fcb7492f2b9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:51.287195 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d2312da2-a683-4b72-a61e-693f667271c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7451c691e50a4d4ab4c33fcb7492f2b9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:51.288704 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:51.297614 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 970/1938] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:12:51 2022] DELETE /identity/v3/users/7451c691e50a4d4ab4c33fcb7492f2b9 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:51.676106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9fad8-88b1-463b-bd33-89cc51d257aa admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:51.677645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0bd9fad8-88b1-463b-bd33-89cc51d257aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:51.691662 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9fad8-88b1-463b-bd33-89cc51d257aa admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:51.705166 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:51.779702 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9a7603-1668-4a09-b1ab-751030eca21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:51.780625 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a7603-1668-4a09-b1ab-751030eca21a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:51.781075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a7603-1668-4a09-b1ab-751030eca21a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:51.781562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9a7603-1668-4a09-b1ab-751030eca21a admin admin] PATH_INFO: `/v3/projects/568db71cc131464eb7b7060444ffe2e7` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:51.782771 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a7603-1668-4a09-b1ab-751030eca21a admin admin] RBAC: Authorizing `identity:delete_project(project_id=568db71cc131464eb7b7060444ffe2e7)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:51.784642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9a7603-1668-4a09-b1ab-751030eca21a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:51.838084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bb9a7603-1668-4a09-b1ab-751030eca21a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '568db71cc131464eb7b7060444ffe2e7', 'request_id': 'req-bb9a7603-1668-4a09-b1ab-751030eca21a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:51.845974 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bb9a7603-1668-4a09-b1ab-751030eca21a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '568db71cc131464eb7b7060444ffe2e7', 'request_id': 'req-bb9a7603-1668-4a09-b1ab-751030eca21a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:51.851407 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bb9a7603-1668-4a09-b1ab-751030eca21a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '568db71cc131464eb7b7060444ffe2e7', 'request_id': 'req-bb9a7603-1668-4a09-b1ab-751030eca21a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:51.862736 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bb9a7603-1668-4a09-b1ab-751030eca21a admin admin] The token cache is being invalidate because project 568db71cc131464eb7b7060444ffe2e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:12:51.863026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bb9a7603-1668-4a09-b1ab-751030eca21a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:51.863655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bb9a7603-1668-4a09-b1ab-751030eca21a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:12:51.865128 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 969/1939] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:12:51 2022] DELETE /identity/v3/projects/568db71cc131464eb7b7060444ffe2e7 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:51.871093 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa97da-e38a-4599-9bd6-86848f1d7275 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:51.871384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa97da-e38a-4599-9bd6-86848f1d7275 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:51.871696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afa97da-e38a-4599-9bd6-86848f1d7275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:51.891342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7afa97da-e38a-4599-9bd6-86848f1d7275 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:51.892541 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7afa97da-e38a-4599-9bd6-86848f1d7275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:51.949533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 971/1940] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:51.958951 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94336625-04d3-4241-a0fa-646431393771 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:51.959231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94336625-04d3-4241-a0fa-646431393771 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:51.959502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94336625-04d3-4241-a0fa-646431393771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:51.979654 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-94336625-04d3-4241-a0fa-646431393771 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:51.980933 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-94336625-04d3-4241-a0fa-646431393771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:51.989069 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 970/1941] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:51.993260 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa97da-e38a-4599-9bd6-86848f1d7275 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:52.004205 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afa97da-e38a-4599-9bd6-86848f1d7275 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:52.008766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d431740-17d9-4e86-aa58-31042e82fe14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:52.009866 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d431740-17d9-4e86-aa58-31042e82fe14 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:52.010138 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d431740-17d9-4e86-aa58-31042e82fe14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:52.010405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d431740-17d9-4e86-aa58-31042e82fe14 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:52.011031 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d431740-17d9-4e86-aa58-31042e82fe14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:12:52.011324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d431740-17d9-4e86-aa58-31042e82fe14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:52.012891 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d431740-17d9-4e86-aa58-31042e82fe14 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:52.024200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 972/1942] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:12:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:12:52.029007 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94336625-04d3-4241-a0fa-646431393771 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:52.038210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94336625-04d3-4241-a0fa-646431393771 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:52.043143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf55e71-66a1-426d-8cae-ad32e48adaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:52.043758 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf55e71-66a1-426d-8cae-ad32e48adaa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:52.044035 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf55e71-66a1-426d-8cae-ad32e48adaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:52.044301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf55e71-66a1-426d-8cae-ad32e48adaa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:52.044925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf55e71-66a1-426d-8cae-ad32e48adaa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:52.046882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf55e71-66a1-426d-8cae-ad32e48adaa5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:52.074495 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 971/1943] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:12:52 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:52.078569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d431740-17d9-4e86-aa58-31042e82fe14 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:52.088657 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d431740-17d9-4e86-aa58-31042e82fe14 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:52.147439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65813446-c299-42e4-bbcf-4980c0b0b69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:52.148053 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65813446-c299-42e4-bbcf-4980c0b0b69c admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:52.148336 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65813446-c299-42e4-bbcf-4980c0b0b69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:52.148608 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65813446-c299-42e4-bbcf-4980c0b0b69c admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:52.149253 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65813446-c299-42e4-bbcf-4980c0b0b69c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:52.151365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65813446-c299-42e4-bbcf-4980c0b0b69c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:52.172081 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 973/1944] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:12:52 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:12:52.176334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf55e71-66a1-426d-8cae-ad32e48adaa5 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:52.187066 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf55e71-66a1-426d-8cae-ad32e48adaa5 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:52.192784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7803217-fc91-4ca6-adb7-8e6e8faf8b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:52.193408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7803217-fc91-4ca6-adb7-8e6e8faf8b10 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:52.193684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7803217-fc91-4ca6-adb7-8e6e8faf8b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:52.193952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7803217-fc91-4ca6-adb7-8e6e8faf8b10 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:52.194597 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7803217-fc91-4ca6-adb7-8e6e8faf8b10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:52.196169 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7803217-fc91-4ca6-adb7-8e6e8faf8b10 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:52.207063 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 972/1945] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:12:52.210965 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65813446-c299-42e4-bbcf-4980c0b0b69c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:52.221345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65813446-c299-42e4-bbcf-4980c0b0b69c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:52.226292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760dde5-29da-4daa-8a89-434b7f85ffad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:52.226937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760dde5-29da-4daa-8a89-434b7f85ffad admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:52.227216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760dde5-29da-4daa-8a89-434b7f85ffad admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:52.227489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760dde5-29da-4daa-8a89-434b7f85ffad admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:52.228247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760dde5-29da-4daa-8a89-434b7f85ffad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:52.229920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760dde5-29da-4daa-8a89-434b7f85ffad admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:52.241408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 974/1946] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:12:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:12:52.246108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7803217-fc91-4ca6-adb7-8e6e8faf8b10 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:52.256484 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7803217-fc91-4ca6-adb7-8e6e8faf8b10 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:52.265518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd32cd-c8c2-4beb-87ed-65e6689d5cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:52.266234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd32cd-c8c2-4beb-87ed-65e6689d5cc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:52.266571 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd32cd-c8c2-4beb-87ed-65e6689d5cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:52.266910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd32cd-c8c2-4beb-87ed-65e6689d5cc7 admin admin] PATH_INFO: `/v3/projects/891e4951e79c40cc89e640d4aa1c3dd8/users/5e7c7d232fa9470b868bcf60a8a35d88/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:52.282053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd32cd-c8c2-4beb-87ed-65e6689d5cc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=891e4951e79c40cc89e640d4aa1c3dd8, user_id=5e7c7d232fa9470b868bcf60a8a35d88, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:52.284728 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd32cd-c8c2-4beb-87ed-65e6689d5cc7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:52.297546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 973/1947] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:12:52 2022] PUT /identity/v3/projects/891e4951e79c40cc89e640d4aa1c3dd8/users/5e7c7d232fa9470b868bcf60a8a35d88/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:12:52.308998 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b24a7-94bf-430a-b2d6-475eb55d435a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:52.309320 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b24a7-94bf-430a-b2d6-475eb55d435a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:52.309596 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1b24a7-94bf-430a-b2d6-475eb55d435a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:52.337602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ef1b24a7-94bf-430a-b2d6-475eb55d435a None None] MFA Rules not processed for user `5e7c7d232fa9470b868bcf60a8a35d88`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:12:52.338912 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ef1b24a7-94bf-430a-b2d6-475eb55d435a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:52.399933 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:52.411232 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 975/1948] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:12:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:12:52.423175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd32cd-c8c2-4beb-87ed-65e6689d5cc7 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:12:52.424953 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a1dd32cd-c8c2-4beb-87ed-65e6689d5cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:12:52.472954 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:12:52.508268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd32cd-c8c2-4beb-87ed-65e6689d5cc7 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:12:52.511866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f335787-30b2-4c42-9bd0-f311fe396a5c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1863336fecad4987acdfd0b3f7b2f70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:12:52.512464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f335787-30b2-4c42-9bd0-f311fe396a5c service glance] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:12:52.512750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f335787-30b2-4c42-9bd0-f311fe396a5c service glance] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:12:52.513069 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f335787-30b2-4c42-9bd0-f311fe396a5c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:12:52.521175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f335787-30b2-4c42-9bd0-f311fe396a5c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:12:52.522605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f335787-30b2-4c42-9bd0-f311fe396a5c service glance] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:12:52.531228 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 974/1949] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:12:52 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:01.760120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b24a7-94bf-430a-b2d6-475eb55d435a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:01.761703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ef1b24a7-94bf-430a-b2d6-475eb55d435a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:01.831071 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1b24a7-94bf-430a-b2d6-475eb55d435a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:01.836712 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d78f1b-ffad-4d99-86ff-54f608813442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:01.837502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d78f1b-ffad-4d99-86ff-54f608813442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:01.837845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d78f1b-ffad-4d99-86ff-54f608813442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:01.838173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d78f1b-ffad-4d99-86ff-54f608813442 admin admin] PATH_INFO: `/v3/users/bd81c673b97648ba91fcdceddf72a828` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:01.839512 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d78f1b-ffad-4d99-86ff-54f608813442 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd81c673b97648ba91fcdceddf72a828)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:01.841493 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d78f1b-ffad-4d99-86ff-54f608813442 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:01.888427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d6d78f1b-ffad-4d99-86ff-54f608813442 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd81c673b97648ba91fcdceddf72a828'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:01.895583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d6d78f1b-ffad-4d99-86ff-54f608813442 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd81c673b97648ba91fcdceddf72a828'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:01.904671 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d6d78f1b-ffad-4d99-86ff-54f608813442 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd81c673b97648ba91fcdceddf72a828'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:01.905382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d6d78f1b-ffad-4d99-86ff-54f608813442 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd81c673b97648ba91fcdceddf72a828'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:01.918532 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d6d78f1b-ffad-4d99-86ff-54f608813442 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd81c673b97648ba91fcdceddf72a828'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:01.924589 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 976/1950] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:01 2022] DELETE /identity/v3/users/bd81c673b97648ba91fcdceddf72a828 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:01.929151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c32ff6-a531-4a97-a21e-bec11592a19d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:01.929673 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c32ff6-a531-4a97-a21e-bec11592a19d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:01.929762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c32ff6-a531-4a97-a21e-bec11592a19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:01.948902 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a8c32ff6-a531-4a97-a21e-bec11592a19d None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:01.950296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a8c32ff6-a531-4a97-a21e-bec11592a19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:02.011864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 975/1951] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:13:02.021472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d78f1b-ffad-4d99-86ff-54f608813442 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:02.022856 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d6d78f1b-ffad-4d99-86ff-54f608813442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:02.091410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d78f1b-ffad-4d99-86ff-54f608813442 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:02.095802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6562c-4c6c-4333-9f34-636abf99ae67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:02.096412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6562c-4c6c-4333-9f34-636abf99ae67 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:02.096698 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6562c-4c6c-4333-9f34-636abf99ae67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:02.096967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6562c-4c6c-4333-9f34-636abf99ae67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:02.106582 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6562c-4c6c-4333-9f34-636abf99ae67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:02.108000 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea6562c-4c6c-4333-9f34-636abf99ae67 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:02.117555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 977/1952] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:02.422201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c32ff6-a531-4a97-a21e-bec11592a19d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:02.423834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a8c32ff6-a531-4a97-a21e-bec11592a19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:02.437037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c32ff6-a531-4a97-a21e-bec11592a19d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:02.442215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:02.443175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:02.443677 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:02.444116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36 admin admin] PATH_INFO: `/v3/projects/4a6741ed6c3348f499654726f04d0b4a` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:02.445729 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a6741ed6c3348f499654726f04d0b4a)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:02.451776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:02.530637 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a6741ed6c3348f499654726f04d0b4a', 'request_id': 'req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:02.539074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a6741ed6c3348f499654726f04d0b4a', 'request_id': 'req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:02.546539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a6741ed6c3348f499654726f04d0b4a', 'request_id': 'req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:02.566232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36 admin admin] The token cache is being invalidate because project 4a6741ed6c3348f499654726f04d0b4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:02.566538 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:02.567180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:02.568697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 976/1953] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:02 2022] DELETE /identity/v3/projects/4a6741ed6c3348f499654726f04d0b4a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:02.573472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6562c-4c6c-4333-9f34-636abf99ae67 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:02.584473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6562c-4c6c-4333-9f34-636abf99ae67 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:02.596924 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:02.647080 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6f408a-6859-4c78-9da6-d98ba79a0d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:02.647777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f408a-6859-4c78-9da6-d98ba79a0d73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:02.648088 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f408a-6859-4c78-9da6-d98ba79a0d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:02.648365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6f408a-6859-4c78-9da6-d98ba79a0d73 admin admin] PATH_INFO: `/v3/users/52d68c02d7b74897add3e84e6c468463` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:02.649544 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f408a-6859-4c78-9da6-d98ba79a0d73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52d68c02d7b74897add3e84e6c468463)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:02.651486 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6f408a-6859-4c78-9da6-d98ba79a0d73 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:02.701597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9f6f408a-6859-4c78-9da6-d98ba79a0d73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52d68c02d7b74897add3e84e6c468463'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:02.708363 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9f6f408a-6859-4c78-9da6-d98ba79a0d73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52d68c02d7b74897add3e84e6c468463'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:02.715300 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9f6f408a-6859-4c78-9da6-d98ba79a0d73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52d68c02d7b74897add3e84e6c468463'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:02.716034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9f6f408a-6859-4c78-9da6-d98ba79a0d73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52d68c02d7b74897add3e84e6c468463'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:02.728179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9f6f408a-6859-4c78-9da6-d98ba79a0d73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52d68c02d7b74897add3e84e6c468463'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:02.734200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 978/1954] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:02 2022] DELETE /identity/v3/users/52d68c02d7b74897add3e84e6c468463 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:13:03.086045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:03.088018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:03.103679 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5f7cb-ef6d-4080-9b30-004b5d1dcb36 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:03.155092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:03.169419 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5844512d-f0e3-4bf6-af9f-ae4c3a5b626c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:03.170050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844512d-f0e3-4bf6-af9f-ae4c3a5b626c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:03.170330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844512d-f0e3-4bf6-af9f-ae4c3a5b626c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:03.170601 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5844512d-f0e3-4bf6-af9f-ae4c3a5b626c admin admin] PATH_INFO: `/v3/projects/e5712a809c2d4b5bbd81bd45e9d51074` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:03.171714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844512d-f0e3-4bf6-af9f-ae4c3a5b626c admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5712a809c2d4b5bbd81bd45e9d51074)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:03.173652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5844512d-f0e3-4bf6-af9f-ae4c3a5b626c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:03.218519 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5844512d-f0e3-4bf6-af9f-ae4c3a5b626c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5712a809c2d4b5bbd81bd45e9d51074', 'request_id': 'req-5844512d-f0e3-4bf6-af9f-ae4c3a5b626c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:03.226060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5844512d-f0e3-4bf6-af9f-ae4c3a5b626c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5712a809c2d4b5bbd81bd45e9d51074', 'request_id': 'req-5844512d-f0e3-4bf6-af9f-ae4c3a5b626c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:03.232457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5844512d-f0e3-4bf6-af9f-ae4c3a5b626c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5712a809c2d4b5bbd81bd45e9d51074', 'request_id': 'req-5844512d-f0e3-4bf6-af9f-ae4c3a5b626c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:03.244024 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5844512d-f0e3-4bf6-af9f-ae4c3a5b626c admin admin] The token cache is being invalidate because project e5712a809c2d4b5bbd81bd45e9d51074 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:03.244310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5844512d-f0e3-4bf6-af9f-ae4c3a5b626c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:03.244942 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5844512d-f0e3-4bf6-af9f-ae4c3a5b626c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:03.246519 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 977/1955] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:03 2022] DELETE /identity/v3/projects/e5712a809c2d4b5bbd81bd45e9d51074 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:03.251942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259384ed-5d0d-48b9-ae3a-04db29de181e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:03.252408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259384ed-5d0d-48b9-ae3a-04db29de181e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:03.252527 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259384ed-5d0d-48b9-ae3a-04db29de181e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:03.271799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-259384ed-5d0d-48b9-ae3a-04db29de181e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:03.272984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-259384ed-5d0d-48b9-ae3a-04db29de181e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:03.332125 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 979/1956] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:03.340561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd176d0-ce8d-462a-8c46-42514341da6c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:03.340786 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd176d0-ce8d-462a-8c46-42514341da6c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:03.341064 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd176d0-ce8d-462a-8c46-42514341da6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:03.360398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4bd176d0-ce8d-462a-8c46-42514341da6c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:03.361635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4bd176d0-ce8d-462a-8c46-42514341da6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:03.370443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 978/1957] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:03.374409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259384ed-5d0d-48b9-ae3a-04db29de181e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:03.384569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259384ed-5d0d-48b9-ae3a-04db29de181e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:03.389215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8114d0-c3d3-4e93-bf74-e557ff4d5906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:03.389807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8114d0-c3d3-4e93-bf74-e557ff4d5906 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:03.390084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8114d0-c3d3-4e93-bf74-e557ff4d5906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:03.390356 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8114d0-c3d3-4e93-bf74-e557ff4d5906 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:03.391001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8114d0-c3d3-4e93-bf74-e557ff4d5906 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:13:03.391288 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8114d0-c3d3-4e93-bf74-e557ff4d5906 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:03.392905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8114d0-c3d3-4e93-bf74-e557ff4d5906 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:03.404809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 980/1958] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:13:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:13:03.408924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd176d0-ce8d-462a-8c46-42514341da6c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:03.418133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd176d0-ce8d-462a-8c46-42514341da6c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:03.423923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6358194-52ec-48b1-b562-0e532cdd1dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:03.424555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6358194-52ec-48b1-b562-0e532cdd1dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:03.424832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6358194-52ec-48b1-b562-0e532cdd1dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:03.425097 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6358194-52ec-48b1-b562-0e532cdd1dc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:03.425771 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6358194-52ec-48b1-b562-0e532cdd1dc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:03.427683 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6358194-52ec-48b1-b562-0e532cdd1dc7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:03.455194 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 979/1959] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:03 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:03.459404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8114d0-c3d3-4e93-bf74-e557ff4d5906 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:03.470211 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8114d0-c3d3-4e93-bf74-e557ff4d5906 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:03.489698 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:03.540505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e000801-1fb6-487a-b65b-9aedf760065c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:03.541113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e000801-1fb6-487a-b65b-9aedf760065c admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:03.541434 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e000801-1fb6-487a-b65b-9aedf760065c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:03.541707 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e000801-1fb6-487a-b65b-9aedf760065c admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:03.542332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e000801-1fb6-487a-b65b-9aedf760065c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:03.544237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e000801-1fb6-487a-b65b-9aedf760065c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:03.569124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 981/1960] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:03 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:03.573879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6358194-52ec-48b1-b562-0e532cdd1dc7 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:03.584889 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6358194-52ec-48b1-b562-0e532cdd1dc7 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:03.592161 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744bbcf7-89d9-4ecb-a3e5-4a29ad4345c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:03.593008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bbcf7-89d9-4ecb-a3e5-4a29ad4345c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:03.593419 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bbcf7-89d9-4ecb-a3e5-4a29ad4345c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:03.593778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744bbcf7-89d9-4ecb-a3e5-4a29ad4345c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:03.594547 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744bbcf7-89d9-4ecb-a3e5-4a29ad4345c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:03.596204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744bbcf7-89d9-4ecb-a3e5-4a29ad4345c9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:03.610769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 980/1961] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:03.617447 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e000801-1fb6-487a-b65b-9aedf760065c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:03.628097 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e000801-1fb6-487a-b65b-9aedf760065c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:03.632732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665e6cda-a3b2-4f56-b07f-9754c1bde03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:03.633397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e6cda-a3b2-4f56-b07f-9754c1bde03e admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:03.634174 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e6cda-a3b2-4f56-b07f-9754c1bde03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:03.634453 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665e6cda-a3b2-4f56-b07f-9754c1bde03e admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:03.635090 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e6cda-a3b2-4f56-b07f-9754c1bde03e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:03.636679 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665e6cda-a3b2-4f56-b07f-9754c1bde03e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:03.647615 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 982/1962] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:03.652698 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744bbcf7-89d9-4ecb-a3e5-4a29ad4345c9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:03.662852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744bbcf7-89d9-4ecb-a3e5-4a29ad4345c9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:03.668139 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8fa19-d22e-4784-96bb-7bf0671e0a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:03.668793 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8fa19-d22e-4784-96bb-7bf0671e0a9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:03.669077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8fa19-d22e-4784-96bb-7bf0671e0a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:03.669389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe8fa19-d22e-4784-96bb-7bf0671e0a9b admin admin] PATH_INFO: `/v3/projects/0456da408bab448fbd6b16a09f7badd9/users/f617dc8aec3a47b9bab93c5776cea215/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:03.687358 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8fa19-d22e-4784-96bb-7bf0671e0a9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0456da408bab448fbd6b16a09f7badd9, user_id=f617dc8aec3a47b9bab93c5776cea215, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:03.690037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe8fa19-d22e-4784-96bb-7bf0671e0a9b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:03.704617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 981/1963] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:03 2022] PUT /identity/v3/projects/0456da408bab448fbd6b16a09f7badd9/users/f617dc8aec3a47b9bab93c5776cea215/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:03.709831 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff18efb-3620-4dd3-8a6d-8b1523b6dec2 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:03.710124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff18efb-3620-4dd3-8a6d-8b1523b6dec2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:03.710416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff18efb-3620-4dd3-8a6d-8b1523b6dec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:03.730118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-1ff18efb-3620-4dd3-8a6d-8b1523b6dec2 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:03.731320 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1ff18efb-3620-4dd3-8a6d-8b1523b6dec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:03.797818 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 983/1964] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:13:03.808270 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8fa19-d22e-4784-96bb-7bf0671e0a9b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:03.809986 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cfe8fa19-d22e-4784-96bb-7bf0671e0a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:03.868542 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe8fa19-d22e-4784-96bb-7bf0671e0a9b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:03.872206 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959b263-8e3d-4733-8b69-499bd507676f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:03.872807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959b263-8e3d-4733-8b69-499bd507676f service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:03.873083 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959b263-8e3d-4733-8b69-499bd507676f service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:03.873391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9959b263-8e3d-4733-8b69-499bd507676f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:03.883894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9959b263-8e3d-4733-8b69-499bd507676f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:03.885388 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9959b263-8e3d-4733-8b69-499bd507676f service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:03.895367 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 982/1965] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:04.627940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ae054-b5ee-4ddf-98ff-d65092e3c264 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:04.627940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ae054-b5ee-4ddf-98ff-d65092e3c264 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:04.628601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77ae054-b5ee-4ddf-98ff-d65092e3c264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:04.671763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c77ae054-b5ee-4ddf-98ff-d65092e3c264 None None] MFA Rules not processed for user `f617dc8aec3a47b9bab93c5776cea215`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:04.674214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c77ae054-b5ee-4ddf-98ff-d65092e3c264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:04.777741 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 984/1966] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:13:04.782265 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959b263-8e3d-4733-8b69-499bd507676f service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:04.796836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9959b263-8e3d-4733-8b69-499bd507676f service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:04.801816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb821aec-5816-4b3f-a9ea-b7adef69497e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:04.802245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb821aec-5816-4b3f-a9ea-b7adef69497e admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:04.802533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb821aec-5816-4b3f-a9ea-b7adef69497e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:04.802843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb821aec-5816-4b3f-a9ea-b7adef69497e admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:04.803575 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb821aec-5816-4b3f-a9ea-b7adef69497e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:04.807801 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb821aec-5816-4b3f-a9ea-b7adef69497e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:04.825135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:04.847573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 983/1967] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:04 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:04.851171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ae054-b5ee-4ddf-98ff-d65092e3c264 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:04.874503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77ae054-b5ee-4ddf-98ff-d65092e3c264 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:04.916593 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:04.950884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea345a9-d6c0-4c81-8862-987b149b9eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:04.951501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea345a9-d6c0-4c81-8862-987b149b9eab admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:04.951783 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea345a9-d6c0-4c81-8862-987b149b9eab admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:04.952059 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea345a9-d6c0-4c81-8862-987b149b9eab admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:04.952688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea345a9-d6c0-4c81-8862-987b149b9eab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:04.957225 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea345a9-d6c0-4c81-8862-987b149b9eab admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:04.979333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 985/1968] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:04 2022] POST /identity/v3/users => generated 292 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:04.984215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb821aec-5816-4b3f-a9ea-b7adef69497e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:04.995304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb821aec-5816-4b3f-a9ea-b7adef69497e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:05.000496 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209bd3b-b8a7-448a-ac30-a6ce38448b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:05.001107 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209bd3b-b8a7-448a-ac30-a6ce38448b06 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:05.001416 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209bd3b-b8a7-448a-ac30-a6ce38448b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:05.001691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e209bd3b-b8a7-448a-ac30-a6ce38448b06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:05.002329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e209bd3b-b8a7-448a-ac30-a6ce38448b06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:05.003921 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e209bd3b-b8a7-448a-ac30-a6ce38448b06 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:05.015742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 984/1969] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:13:05.019883 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea345a9-d6c0-4c81-8862-987b149b9eab admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:05.031226 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea345a9-d6c0-4c81-8862-987b149b9eab admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:05.036467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd98d7d-2505-43d1-ba2f-22c64c3842b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:05.037138 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd98d7d-2505-43d1-ba2f-22c64c3842b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:05.037461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd98d7d-2505-43d1-ba2f-22c64c3842b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:05.037764 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd98d7d-2505-43d1-ba2f-22c64c3842b5 admin admin] PATH_INFO: `/v3/projects/d08e4234cb56400696801d46990f33b2/users/e601e3391db64336b8b8af3010c93bd4/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:05.053106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd98d7d-2505-43d1-ba2f-22c64c3842b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d08e4234cb56400696801d46990f33b2, user_id=e601e3391db64336b8b8af3010c93bd4, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:05.055825 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd98d7d-2505-43d1-ba2f-22c64c3842b5 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:05.069007 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 986/1970] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:05 2022] PUT /identity/v3/projects/d08e4234cb56400696801d46990f33b2/users/e601e3391db64336b8b8af3010c93bd4/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:05.073638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209bd3b-b8a7-448a-ac30-a6ce38448b06 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:05.083884 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e209bd3b-b8a7-448a-ac30-a6ce38448b06 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:05.140261 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88245e4c-e8a6-4a50-bc04-fdd786f36a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:05.140783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88245e4c-e8a6-4a50-bc04-fdd786f36a86 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:05.141061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88245e4c-e8a6-4a50-bc04-fdd786f36a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:05.141363 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88245e4c-e8a6-4a50-bc04-fdd786f36a86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:05.141992 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88245e4c-e8a6-4a50-bc04-fdd786f36a86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:05.143566 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88245e4c-e8a6-4a50-bc04-fdd786f36a86 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:05.153130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 985/1971] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:05.157393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd98d7d-2505-43d1-ba2f-22c64c3842b5 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:05.168397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd98d7d-2505-43d1-ba2f-22c64c3842b5 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:05.173091 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ecbb9-7857-439d-b895-cbc6dfce9804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:05.173765 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ecbb9-7857-439d-b895-cbc6dfce9804 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:05.174040 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ecbb9-7857-439d-b895-cbc6dfce9804 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:05.174337 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ecbb9-7857-439d-b895-cbc6dfce9804 admin admin] PATH_INFO: `/v3/projects/d08e4234cb56400696801d46990f33b2/users/e601e3391db64336b8b8af3010c93bd4/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:05.176188 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ecbb9-7857-439d-b895-cbc6dfce9804 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d08e4234cb56400696801d46990f33b2, user_id=e601e3391db64336b8b8af3010c93bd4, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:05.178818 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ecbb9-7857-439d-b895-cbc6dfce9804 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:05.189269 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 987/1972] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:05 2022] PUT /identity/v3/projects/d08e4234cb56400696801d46990f33b2/users/e601e3391db64336b8b8af3010c93bd4/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:05.979345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f8808-7251-4720-870d-d3201b1260b3 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:05.979906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f8808-7251-4720-870d-d3201b1260b3 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:05.979992 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f8808-7251-4720-870d-d3201b1260b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:06.025697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-cb8f8808-7251-4720-870d-d3201b1260b3 None None] MFA Rules not processed for user `e601e3391db64336b8b8af3010c93bd4`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:06.027141 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cb8f8808-7251-4720-870d-d3201b1260b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:06.145442 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 986/1973] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:05 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:06.157906 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ecbb9-7857-439d-b895-cbc6dfce9804 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:06.159790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3d4ecbb9-7857-439d-b895-cbc6dfce9804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:06.236660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ecbb9-7857-439d-b895-cbc6dfce9804 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:06.240770 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688df09f-3e8c-439a-b655-968890b1c341 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:06.241604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688df09f-3e8c-439a-b655-968890b1c341 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:06.242003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688df09f-3e8c-439a-b655-968890b1c341 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:06.242551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688df09f-3e8c-439a-b655-968890b1c341 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:06.347177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688df09f-3e8c-439a-b655-968890b1c341 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:06.348603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688df09f-3e8c-439a-b655-968890b1c341 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:06.357765 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 988/1974] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:15.993464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51e87b-b206-4b64-b12f-c1d952a03bef None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:15.994039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51e87b-b206-4b64-b12f-c1d952a03bef None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:15.994039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51e87b-b206-4b64-b12f-c1d952a03bef None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:15.995542 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 987/1975] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 19:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:13:16.014064 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163342b-a393-4f91-9484-9e1adf5e9d06 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:16.015281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163342b-a393-4f91-9484-9e1adf5e9d06 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:16.015281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163342b-a393-4f91-9484-9e1adf5e9d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:16.051080 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4163342b-a393-4f91-9484-9e1adf5e9d06 None None] MFA Rules not processed for user `673a72ac9a9847e3812007b66ac7a5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:16.052563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4163342b-a393-4f91-9484-9e1adf5e9d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:16.100550 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:16.142598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=70 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:16.172070 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 989/1976] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 19:13:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:16.185950 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51e87b-b206-4b64-b12f-c1d952a03bef None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:16.191980 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8f51e87b-b206-4b64-b12f-c1d952a03bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:16.222431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51e87b-b206-4b64-b12f-c1d952a03bef None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:16.231991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850bcd8-342e-43ff-b18b-f6bb5d0304e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:16.232689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850bcd8-342e-43ff-b18b-f6bb5d0304e8 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:16.233995 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850bcd8-342e-43ff-b18b-f6bb5d0304e8 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:16.235533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1850bcd8-342e-43ff-b18b-f6bb5d0304e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:16.253722 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850bcd8-342e-43ff-b18b-f6bb5d0304e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:16.253722 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1850bcd8-342e-43ff-b18b-f6bb5d0304e8 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:16.272530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 988/1977] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:17.274141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163342b-a393-4f91-9484-9e1adf5e9d06 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:17.276240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4163342b-a393-4f91-9484-9e1adf5e9d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:17.287902 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163342b-a393-4f91-9484-9e1adf5e9d06 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:17.292328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d14e6-efa1-4130-b96d-e34c07fcc54a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:17.293088 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d14e6-efa1-4130-b96d-e34c07fcc54a service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:17.293436 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d14e6-efa1-4130-b96d-e34c07fcc54a service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:17.293806 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d14e6-efa1-4130-b96d-e34c07fcc54a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:17.307555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d14e6-efa1-4130-b96d-e34c07fcc54a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:17.309460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9d14e6-efa1-4130-b96d-e34c07fcc54a service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:17.320594 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 990/1978] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:23.694596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850bcd8-342e-43ff-b18b-f6bb5d0304e8 service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:23.704782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1850bcd8-342e-43ff-b18b-f6bb5d0304e8 service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:23.709080 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45f0dd6-5ec4-4687-94d7-e2656e423434 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:23.709717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f0dd6-5ec4-4687-94d7-e2656e423434 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:23.709989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f0dd6-5ec4-4687-94d7-e2656e423434 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:23.710257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45f0dd6-5ec4-4687-94d7-e2656e423434 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:23.713153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e45f0dd6-5ec4-4687-94d7-e2656e423434 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:23.814573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f0dd6-5ec4-4687-94d7-e2656e423434 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:23.816856 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45f0dd6-5ec4-4687-94d7-e2656e423434 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:23.833946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 989/1979] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:23 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:27.315760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=20 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:28.896103 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d14e6-efa1-4130-b96d-e34c07fcc54a service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:28.897815 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-da9d14e6-efa1-4130-b96d-e34c07fcc54a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:28.991679 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d14e6-efa1-4130-b96d-e34c07fcc54a service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:28.999790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0384e6-c244-4e6f-a0cf-2abb85d94546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:29.000787 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0384e6-c244-4e6f-a0cf-2abb85d94546 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:29.001250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0384e6-c244-4e6f-a0cf-2abb85d94546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:29.001716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0384e6-c244-4e6f-a0cf-2abb85d94546 admin admin] PATH_INFO: `/v3/users/451b44b6d3eb48c9b035db2091953a62` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:29.003095 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0384e6-c244-4e6f-a0cf-2abb85d94546 admin admin] RBAC: Authorizing `identity:delete_user(user_id=451b44b6d3eb48c9b035db2091953a62)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:29.005089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0384e6-c244-4e6f-a0cf-2abb85d94546 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:29.092035 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-4b0384e6-c244-4e6f-a0cf-2abb85d94546 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '451b44b6d3eb48c9b035db2091953a62'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:29.101658 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-4b0384e6-c244-4e6f-a0cf-2abb85d94546 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '451b44b6d3eb48c9b035db2091953a62'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:29.111677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-4b0384e6-c244-4e6f-a0cf-2abb85d94546 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '451b44b6d3eb48c9b035db2091953a62'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:29.112419 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-4b0384e6-c244-4e6f-a0cf-2abb85d94546 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '451b44b6d3eb48c9b035db2091953a62'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:29.126970 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-4b0384e6-c244-4e6f-a0cf-2abb85d94546 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '451b44b6d3eb48c9b035db2091953a62'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:29.137325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 991/1980] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:28 2022] DELETE /identity/v3/users/451b44b6d3eb48c9b035db2091953a62 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:29.140184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2138e17-8829-4d0e-89e0-e6751e770dc4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:29.140691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2138e17-8829-4d0e-89e0-e6751e770dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:29.140784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2138e17-8829-4d0e-89e0-e6751e770dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:29.169542 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-d2138e17-8829-4d0e-89e0-e6751e770dc4 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:29.170830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d2138e17-8829-4d0e-89e0-e6751e770dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:29.251657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 990/1981] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:29.261404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0384e6-c244-4e6f-a0cf-2abb85d94546 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:29.262889 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4b0384e6-c244-4e6f-a0cf-2abb85d94546 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:29.319391 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0384e6-c244-4e6f-a0cf-2abb85d94546 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:29.322953 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3e7f0-9bc5-4cad-be42-212543d9c9a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:29.323552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3e7f0-9bc5-4cad-be42-212543d9c9a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:29.323826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3e7f0-9bc5-4cad-be42-212543d9c9a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:29.324092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f3e7f0-9bc5-4cad-be42-212543d9c9a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:29.333773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3e7f0-9bc5-4cad-be42-212543d9c9a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:29.335174 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f3e7f0-9bc5-4cad-be42-212543d9c9a3 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:29.344448 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 992/1982] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:29.618654 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2138e17-8829-4d0e-89e0-e6751e770dc4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:29.620196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d2138e17-8829-4d0e-89e0-e6751e770dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:29.634385 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2138e17-8829-4d0e-89e0-e6751e770dc4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:29.639254 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:29.639898 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:29.640179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:29.640452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7 admin admin] PATH_INFO: `/v3/projects/3061d78415204980bf871fa211f55798` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:29.641604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3061d78415204980bf871fa211f55798)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:29.643498 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:29.692559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3061d78415204980bf871fa211f55798', 'request_id': 'req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:29.699843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3061d78415204980bf871fa211f55798', 'request_id': 'req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:29.705376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3061d78415204980bf871fa211f55798', 'request_id': 'req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:29.716929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7 admin admin] The token cache is being invalidate because project 3061d78415204980bf871fa211f55798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:29.717237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:29.717890 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:29.719382 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 991/1983] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:29 2022] DELETE /identity/v3/projects/3061d78415204980bf871fa211f55798 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:29.723516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3e7f0-9bc5-4cad-be42-212543d9c9a3 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:29.740830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f3e7f0-9bc5-4cad-be42-212543d9c9a3 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:29.765456 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:29.812609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a4bba2-e41f-4a36-910c-40213296e715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:29.813261 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4bba2-e41f-4a36-910c-40213296e715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:29.813540 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4bba2-e41f-4a36-910c-40213296e715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:29.813810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a4bba2-e41f-4a36-910c-40213296e715 admin admin] PATH_INFO: `/v3/users/fb2e6bc90b3a4f198ca5495a32e4faa9` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:29.814909 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4bba2-e41f-4a36-910c-40213296e715 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb2e6bc90b3a4f198ca5495a32e4faa9)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:29.816809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a4bba2-e41f-4a36-910c-40213296e715 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:29.862144 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-88a4bba2-e41f-4a36-910c-40213296e715 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb2e6bc90b3a4f198ca5495a32e4faa9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:29.868771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-88a4bba2-e41f-4a36-910c-40213296e715 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb2e6bc90b3a4f198ca5495a32e4faa9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:29.875544 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-88a4bba2-e41f-4a36-910c-40213296e715 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb2e6bc90b3a4f198ca5495a32e4faa9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:29.876296 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-88a4bba2-e41f-4a36-910c-40213296e715 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb2e6bc90b3a4f198ca5495a32e4faa9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:29.888242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-88a4bba2-e41f-4a36-910c-40213296e715 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb2e6bc90b3a4f198ca5495a32e4faa9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:29.894077 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 993/1984] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:29 2022] DELETE /identity/v3/users/fb2e6bc90b3a4f198ca5495a32e4faa9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:30.264320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:30.265917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:30.278344 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:30.282931 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df095bdb-d5f1-46fb-b21b-6cac7b8c60b7 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:30.362570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c53df67-f7f0-49c6-a1e3-4b1206faa3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:30.365957 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53df67-f7f0-49c6-a1e3-4b1206faa3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:30.366352 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53df67-f7f0-49c6-a1e3-4b1206faa3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:30.366745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53df67-f7f0-49c6-a1e3-4b1206faa3b0 admin admin] PATH_INFO: `/v3/projects/412834d055f14ef8ab3e335f3df414c5` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:30.368158 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53df67-f7f0-49c6-a1e3-4b1206faa3b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=412834d055f14ef8ab3e335f3df414c5)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:30.370855 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c53df67-f7f0-49c6-a1e3-4b1206faa3b0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:30.440191 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9c53df67-f7f0-49c6-a1e3-4b1206faa3b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '412834d055f14ef8ab3e335f3df414c5', 'request_id': 'req-9c53df67-f7f0-49c6-a1e3-4b1206faa3b0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:30.450982 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9c53df67-f7f0-49c6-a1e3-4b1206faa3b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '412834d055f14ef8ab3e335f3df414c5', 'request_id': 'req-9c53df67-f7f0-49c6-a1e3-4b1206faa3b0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:30.460201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9c53df67-f7f0-49c6-a1e3-4b1206faa3b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '412834d055f14ef8ab3e335f3df414c5', 'request_id': 'req-9c53df67-f7f0-49c6-a1e3-4b1206faa3b0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:30.476758 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9c53df67-f7f0-49c6-a1e3-4b1206faa3b0 admin admin] The token cache is being invalidate because project 412834d055f14ef8ab3e335f3df414c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:30.477393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9c53df67-f7f0-49c6-a1e3-4b1206faa3b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:30.478744 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9c53df67-f7f0-49c6-a1e3-4b1206faa3b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:30.480635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 992/1985] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:30 2022] DELETE /identity/v3/projects/412834d055f14ef8ab3e335f3df414c5 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:30.486228 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488d0a1-ecc5-42fc-9827-ee355edacec9 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:30.486773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488d0a1-ecc5-42fc-9827-ee355edacec9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:30.486886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488d0a1-ecc5-42fc-9827-ee355edacec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:30.509084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e488d0a1-ecc5-42fc-9827-ee355edacec9 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:30.510907 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e488d0a1-ecc5-42fc-9827-ee355edacec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:30.583443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 994/1986] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:30.591220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f5488-cf9c-44ee-83e7-42a47548fb27 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:30.591500 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f5488-cf9c-44ee-83e7-42a47548fb27 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:30.591770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218f5488-cf9c-44ee-83e7-42a47548fb27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:30.612681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-218f5488-cf9c-44ee-83e7-42a47548fb27 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:30.613906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-218f5488-cf9c-44ee-83e7-42a47548fb27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:30.623443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 993/1987] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:30.627567 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488d0a1-ecc5-42fc-9827-ee355edacec9 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:30.638130 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488d0a1-ecc5-42fc-9827-ee355edacec9 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:30.642699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c9371-6204-4a4d-ad7c-b59dda49b725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:30.643288 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c9371-6204-4a4d-ad7c-b59dda49b725 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:30.643566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c9371-6204-4a4d-ad7c-b59dda49b725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:30.643837 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c9371-6204-4a4d-ad7c-b59dda49b725 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:30.644470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c9371-6204-4a4d-ad7c-b59dda49b725 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:13:30.644768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c9371-6204-4a4d-ad7c-b59dda49b725 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:30.646375 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c9371-6204-4a4d-ad7c-b59dda49b725 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:30.657805 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 995/1988] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:13:30.662110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f5488-cf9c-44ee-83e7-42a47548fb27 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:30.671574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218f5488-cf9c-44ee-83e7-42a47548fb27 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:30.676162 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb3ede-caf2-47bc-9e12-19f227bace34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:30.676750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb3ede-caf2-47bc-9e12-19f227bace34 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:30.677027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb3ede-caf2-47bc-9e12-19f227bace34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:30.677334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb3ede-caf2-47bc-9e12-19f227bace34 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:30.677955 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb3ede-caf2-47bc-9e12-19f227bace34 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:30.679835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb3ede-caf2-47bc-9e12-19f227bace34 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:30.707602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 994/1989] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:30 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:13:30.711804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c9371-6204-4a4d-ad7c-b59dda49b725 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:30.721777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c9371-6204-4a4d-ad7c-b59dda49b725 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:30.748321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:30.778860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d21f13-c208-4402-bb34-4e4e994cce39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:30.779443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d21f13-c208-4402-bb34-4e4e994cce39 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:30.779714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d21f13-c208-4402-bb34-4e4e994cce39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:30.779977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d21f13-c208-4402-bb34-4e4e994cce39 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:30.780586 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d21f13-c208-4402-bb34-4e4e994cce39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:30.782501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d21f13-c208-4402-bb34-4e4e994cce39 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:30.803213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 996/1990] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:30 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:30.807985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb3ede-caf2-47bc-9e12-19f227bace34 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:30.818153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb3ede-caf2-47bc-9e12-19f227bace34 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:30.822912 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ee1f5-9a8b-40d3-92d6-fe05741eb877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:30.823507 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ee1f5-9a8b-40d3-92d6-fe05741eb877 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:30.823782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ee1f5-9a8b-40d3-92d6-fe05741eb877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:30.824050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2ee1f5-9a8b-40d3-92d6-fe05741eb877 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:30.824687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ee1f5-9a8b-40d3-92d6-fe05741eb877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:30.826290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2ee1f5-9a8b-40d3-92d6-fe05741eb877 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:30.838220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 995/1991] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:30.842312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d21f13-c208-4402-bb34-4e4e994cce39 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:30.853817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d21f13-c208-4402-bb34-4e4e994cce39 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:30.858799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc27b70-7d14-492c-a912-a2e5446b4ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:30.859420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc27b70-7d14-492c-a912-a2e5446b4ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:30.859703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc27b70-7d14-492c-a912-a2e5446b4ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:30.859977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc27b70-7d14-492c-a912-a2e5446b4ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:30.860627 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc27b70-7d14-492c-a912-a2e5446b4ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:30.862260 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc27b70-7d14-492c-a912-a2e5446b4ff0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:30.875033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 997/1992] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:13:30.879343 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ee1f5-9a8b-40d3-92d6-fe05741eb877 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:30.889503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2ee1f5-9a8b-40d3-92d6-fe05741eb877 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:30.894077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cf4ca-8bca-4df9-a56e-268197ebc3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:30.894779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cf4ca-8bca-4df9-a56e-268197ebc3d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:30.895059 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cf4ca-8bca-4df9-a56e-268197ebc3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:30.895329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cf4ca-8bca-4df9-a56e-268197ebc3d9 admin admin] PATH_INFO: `/v3/projects/9bb67a267eff4e48b6e928c31b306433/users/954b0fedb44e4d66a0313dc0e1b094e0/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:30.910697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cf4ca-8bca-4df9-a56e-268197ebc3d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb67a267eff4e48b6e928c31b306433, user_id=954b0fedb44e4d66a0313dc0e1b094e0, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:30.913352 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cf4ca-8bca-4df9-a56e-268197ebc3d9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:30.927189 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 996/1993] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:30 2022] PUT /identity/v3/projects/9bb67a267eff4e48b6e928c31b306433/users/954b0fedb44e4d66a0313dc0e1b094e0/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:30.939410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab51b7-d4c4-43e4-863f-fdbf466ac246 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:30.939690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab51b7-d4c4-43e4-863f-fdbf466ac246 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:30.939958 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab51b7-d4c4-43e4-863f-fdbf466ac246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:30.968382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-dfab51b7-d4c4-43e4-863f-fdbf466ac246 None None] MFA Rules not processed for user `954b0fedb44e4d66a0313dc0e1b094e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:30.969657 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-dfab51b7-d4c4-43e4-863f-fdbf466ac246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:31.052512 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 998/1994] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:31.061173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cf4ca-8bca-4df9-a56e-268197ebc3d9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:31.098366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:31.134656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cf4ca-8bca-4df9-a56e-268197ebc3d9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:31.140287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214c889-5408-435a-8753-c3d059de965a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:31.141020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214c889-5408-435a-8753-c3d059de965a admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:31.141404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214c889-5408-435a-8753-c3d059de965a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:31.141703 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a214c889-5408-435a-8753-c3d059de965a admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:31.142332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214c889-5408-435a-8753-c3d059de965a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:31.144224 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a214c889-5408-435a-8753-c3d059de965a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:31.184035 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 997/1995] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:31 2022] POST /identity/v3/projects => generated 369 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:31.185483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab51b7-d4c4-43e4-863f-fdbf466ac246 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:31.197588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab51b7-d4c4-43e4-863f-fdbf466ac246 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:31.256309 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc048a5b-e903-48cf-96f6-cfa5e2206aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:31.256906 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc048a5b-e903-48cf-96f6-cfa5e2206aec admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:31.257182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc048a5b-e903-48cf-96f6-cfa5e2206aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:31.257479 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc048a5b-e903-48cf-96f6-cfa5e2206aec admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:31.258089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc048a5b-e903-48cf-96f6-cfa5e2206aec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:31.260018 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc048a5b-e903-48cf-96f6-cfa5e2206aec admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:31.282318 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 999/1996] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:31 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:31.286088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214c889-5408-435a-8753-c3d059de965a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:31.297282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a214c889-5408-435a-8753-c3d059de965a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:31.301951 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4fd44-73d8-4f6d-b5b8-00a653bfeb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:31.302663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4fd44-73d8-4f6d-b5b8-00a653bfeb3b admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:31.303030 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4fd44-73d8-4f6d-b5b8-00a653bfeb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:31.303374 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4fd44-73d8-4f6d-b5b8-00a653bfeb3b admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:31.304082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4fd44-73d8-4f6d-b5b8-00a653bfeb3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:31.305774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb4fd44-73d8-4f6d-b5b8-00a653bfeb3b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:31.320305 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 998/1997] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:31.321643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc048a5b-e903-48cf-96f6-cfa5e2206aec admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:31.331835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc048a5b-e903-48cf-96f6-cfa5e2206aec admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:31.336559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31c51d-e137-48a6-bfed-f4f648532d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:31.337740 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31c51d-e137-48a6-bfed-f4f648532d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:31.338019 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31c51d-e137-48a6-bfed-f4f648532d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:31.338318 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31c51d-e137-48a6-bfed-f4f648532d16 admin admin] PATH_INFO: `/v3/projects/09f9dd9f8cfe4b9a82d0d21a6bacb132/users/1ef55dbd0a374f87b70040a1d658fe04/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:31.354828 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31c51d-e137-48a6-bfed-f4f648532d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09f9dd9f8cfe4b9a82d0d21a6bacb132, user_id=1ef55dbd0a374f87b70040a1d658fe04, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:31.357577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31c51d-e137-48a6-bfed-f4f648532d16 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:31.375645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1000/1998] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:31 2022] PUT /identity/v3/projects/09f9dd9f8cfe4b9a82d0d21a6bacb132/users/1ef55dbd0a374f87b70040a1d658fe04/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:13:31.379558 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4fd44-73d8-4f6d-b5b8-00a653bfeb3b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:31.398680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb4fd44-73d8-4f6d-b5b8-00a653bfeb3b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:31.468728 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a4e597-200f-40e4-99a0-2a4fc02ba50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:31.469407 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4e597-200f-40e4-99a0-2a4fc02ba50e admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:31.469786 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4e597-200f-40e4-99a0-2a4fc02ba50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:31.470061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a4e597-200f-40e4-99a0-2a4fc02ba50e admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:31.470691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4e597-200f-40e4-99a0-2a4fc02ba50e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:31.472285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a4e597-200f-40e4-99a0-2a4fc02ba50e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:31.483449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 999/1999] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:31.487948 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31c51d-e137-48a6-bfed-f4f648532d16 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:31.493687 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:31.500308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31c51d-e137-48a6-bfed-f4f648532d16 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:31.506505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df98e7-6b1f-4b1a-a7d6-fd8410dbf030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:31.507167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df98e7-6b1f-4b1a-a7d6-fd8410dbf030 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:31.507447 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df98e7-6b1f-4b1a-a7d6-fd8410dbf030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:31.507716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df98e7-6b1f-4b1a-a7d6-fd8410dbf030 admin admin] PATH_INFO: `/v3/projects/09f9dd9f8cfe4b9a82d0d21a6bacb132/users/1ef55dbd0a374f87b70040a1d658fe04/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:31.510220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df98e7-6b1f-4b1a-a7d6-fd8410dbf030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09f9dd9f8cfe4b9a82d0d21a6bacb132, user_id=1ef55dbd0a374f87b70040a1d658fe04, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:31.513153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df98e7-6b1f-4b1a-a7d6-fd8410dbf030 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:31.534554 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1001/2000] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:31 2022] PUT /identity/v3/projects/09f9dd9f8cfe4b9a82d0d21a6bacb132/users/1ef55dbd0a374f87b70040a1d658fe04/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:31.537533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22160d12-f954-48f7-a467-fdfff82530a6 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:31.537924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22160d12-f954-48f7-a467-fdfff82530a6 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:31.538300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22160d12-f954-48f7-a467-fdfff82530a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:31.573634 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-22160d12-f954-48f7-a467-fdfff82530a6 None None] MFA Rules not processed for user `1ef55dbd0a374f87b70040a1d658fe04`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:31.575150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-22160d12-f954-48f7-a467-fdfff82530a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:31.658999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1000/2001] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:31.670457 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df98e7-6b1f-4b1a-a7d6-fd8410dbf030 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:31.677577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-51df98e7-6b1f-4b1a-a7d6-fd8410dbf030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:31.748694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df98e7-6b1f-4b1a-a7d6-fd8410dbf030 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:31.752320 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d8d4e-9d04-4598-ba9e-3cc77fb74793 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:31.752925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d8d4e-9d04-4598-ba9e-3cc77fb74793 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:31.753250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d8d4e-9d04-4598-ba9e-3cc77fb74793 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:31.753506 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d8d4e-9d04-4598-ba9e-3cc77fb74793 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:31.763060 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d8d4e-9d04-4598-ba9e-3cc77fb74793 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:31.764469 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d8d4e-9d04-4598-ba9e-3cc77fb74793 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:31.773912 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1002/2002] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:31.819766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22160d12-f954-48f7-a467-fdfff82530a6 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:31.821215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-22160d12-f954-48f7-a467-fdfff82530a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:31.833407 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22160d12-f954-48f7-a467-fdfff82530a6 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:31.836926 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe65ed0-8c01-40c4-b710-b91cf0f5ebe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:31.837573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe65ed0-8c01-40c4-b710-b91cf0f5ebe0 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:31.837847 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe65ed0-8c01-40c4-b710-b91cf0f5ebe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:31.838115 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe65ed0-8c01-40c4-b710-b91cf0f5ebe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:31.862700 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:31.895814 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe65ed0-8c01-40c4-b710-b91cf0f5ebe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:31.897281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe65ed0-8c01-40c4-b710-b91cf0f5ebe0 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:31.906231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1001/2003] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:35.255321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d8d4e-9d04-4598-ba9e-3cc77fb74793 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:35.321142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d8d4e-9d04-4598-ba9e-3cc77fb74793 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:35.326336 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869aab43-b7d8-4127-a7e3-7865a4ea7c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:35.326955 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869aab43-b7d8-4127-a7e3-7865a4ea7c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:35.327238 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869aab43-b7d8-4127-a7e3-7865a4ea7c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:35.327512 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869aab43-b7d8-4127-a7e3-7865a4ea7c1e admin admin] PATH_INFO: `/v3/users/954b0fedb44e4d66a0313dc0e1b094e0` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:35.328612 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869aab43-b7d8-4127-a7e3-7865a4ea7c1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=954b0fedb44e4d66a0313dc0e1b094e0)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:35.330639 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869aab43-b7d8-4127-a7e3-7865a4ea7c1e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:35.377133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-869aab43-b7d8-4127-a7e3-7865a4ea7c1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '954b0fedb44e4d66a0313dc0e1b094e0'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:35.393811 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-869aab43-b7d8-4127-a7e3-7865a4ea7c1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '954b0fedb44e4d66a0313dc0e1b094e0'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:35.404355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-869aab43-b7d8-4127-a7e3-7865a4ea7c1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '954b0fedb44e4d66a0313dc0e1b094e0'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:35.404962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-869aab43-b7d8-4127-a7e3-7865a4ea7c1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '954b0fedb44e4d66a0313dc0e1b094e0'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:35.417964 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-869aab43-b7d8-4127-a7e3-7865a4ea7c1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '954b0fedb44e4d66a0313dc0e1b094e0'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:35.424072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1003/2004] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:35 2022] DELETE /identity/v3/users/954b0fedb44e4d66a0313dc0e1b094e0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:35.428661 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd51c1d-0e09-42cd-8b72-9dd82e1200e4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:35.429114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd51c1d-0e09-42cd-8b72-9dd82e1200e4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:35.429114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd51c1d-0e09-42cd-8b72-9dd82e1200e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:35.448812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-7dd51c1d-0e09-42cd-8b72-9dd82e1200e4 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:35.450048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7dd51c1d-0e09-42cd-8b72-9dd82e1200e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:35.508176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1002/2005] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:13:35.517852 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869aab43-b7d8-4127-a7e3-7865a4ea7c1e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:35.519216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-869aab43-b7d8-4127-a7e3-7865a4ea7c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:35.532338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:35.578250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869aab43-b7d8-4127-a7e3-7865a4ea7c1e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:35.581938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77027bbe-6d96-4fdb-b0e2-0f85c15e11ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:35.582548 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77027bbe-6d96-4fdb-b0e2-0f85c15e11ca service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:35.582825 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77027bbe-6d96-4fdb-b0e2-0f85c15e11ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:35.583096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77027bbe-6d96-4fdb-b0e2-0f85c15e11ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:35.592421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77027bbe-6d96-4fdb-b0e2-0f85c15e11ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:35.593872 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77027bbe-6d96-4fdb-b0e2-0f85c15e11ca service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:35.603804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1004/2006] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:35.865341 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd51c1d-0e09-42cd-8b72-9dd82e1200e4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:35.866883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7dd51c1d-0e09-42cd-8b72-9dd82e1200e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:35.879935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd51c1d-0e09-42cd-8b72-9dd82e1200e4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:35.884882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:35.885530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:35.885807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:35.886078 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170 admin admin] PATH_INFO: `/v3/projects/9bb67a267eff4e48b6e928c31b306433` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:35.887183 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bb67a267eff4e48b6e928c31b306433)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:35.889068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:35.938352 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bb67a267eff4e48b6e928c31b306433', 'request_id': 'req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:35.951843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bb67a267eff4e48b6e928c31b306433', 'request_id': 'req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:35.957627 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bb67a267eff4e48b6e928c31b306433', 'request_id': 'req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:35.969073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170 admin admin] The token cache is being invalidate because project 9bb67a267eff4e48b6e928c31b306433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:35.969400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:35.970025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:35.971509 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1003/2007] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:35 2022] DELETE /identity/v3/projects/9bb67a267eff4e48b6e928c31b306433 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:35.975584 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77027bbe-6d96-4fdb-b0e2-0f85c15e11ca service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:35.985315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77027bbe-6d96-4fdb-b0e2-0f85c15e11ca service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:36.043374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bfa28-2ec4-4c13-821b-33a3100f40dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:36.044005 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bfa28-2ec4-4c13-821b-33a3100f40dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:36.044278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bfa28-2ec4-4c13-821b-33a3100f40dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:36.044546 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bfa28-2ec4-4c13-821b-33a3100f40dc admin admin] PATH_INFO: `/v3/users/1ef55dbd0a374f87b70040a1d658fe04` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:36.045685 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bfa28-2ec4-4c13-821b-33a3100f40dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ef55dbd0a374f87b70040a1d658fe04)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:36.047573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bfa28-2ec4-4c13-821b-33a3100f40dc admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:36.101359 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-246bfa28-2ec4-4c13-821b-33a3100f40dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ef55dbd0a374f87b70040a1d658fe04'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:36.111992 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-246bfa28-2ec4-4c13-821b-33a3100f40dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ef55dbd0a374f87b70040a1d658fe04'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:36.123547 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-246bfa28-2ec4-4c13-821b-33a3100f40dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ef55dbd0a374f87b70040a1d658fe04'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:36.124220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-246bfa28-2ec4-4c13-821b-33a3100f40dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ef55dbd0a374f87b70040a1d658fe04'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:36.141460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-246bfa28-2ec4-4c13-821b-33a3100f40dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ef55dbd0a374f87b70040a1d658fe04'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:36.148273 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1005/2008] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:35 2022] DELETE /identity/v3/users/1ef55dbd0a374f87b70040a1d658fe04 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:36.512507 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:36.513998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:36.527279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad2f9cd-7a0a-4536-8e91-968c5b0d2170 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:36.572612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:36.586766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485387bd-c8b1-4aa5-8029-05d7d39df876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:36.587391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485387bd-c8b1-4aa5-8029-05d7d39df876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:36.587668 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485387bd-c8b1-4aa5-8029-05d7d39df876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:36.587937 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485387bd-c8b1-4aa5-8029-05d7d39df876 admin admin] PATH_INFO: `/v3/projects/09f9dd9f8cfe4b9a82d0d21a6bacb132` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:36.589040 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485387bd-c8b1-4aa5-8029-05d7d39df876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09f9dd9f8cfe4b9a82d0d21a6bacb132)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:36.590962 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485387bd-c8b1-4aa5-8029-05d7d39df876 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:36.637907 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-485387bd-c8b1-4aa5-8029-05d7d39df876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09f9dd9f8cfe4b9a82d0d21a6bacb132', 'request_id': 'req-485387bd-c8b1-4aa5-8029-05d7d39df876'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:36.646229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-485387bd-c8b1-4aa5-8029-05d7d39df876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09f9dd9f8cfe4b9a82d0d21a6bacb132', 'request_id': 'req-485387bd-c8b1-4aa5-8029-05d7d39df876'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:36.652126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-485387bd-c8b1-4aa5-8029-05d7d39df876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09f9dd9f8cfe4b9a82d0d21a6bacb132', 'request_id': 'req-485387bd-c8b1-4aa5-8029-05d7d39df876'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:36.662976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-485387bd-c8b1-4aa5-8029-05d7d39df876 admin admin] The token cache is being invalidate because project 09f9dd9f8cfe4b9a82d0d21a6bacb132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:36.663259 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-485387bd-c8b1-4aa5-8029-05d7d39df876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:36.663887 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-485387bd-c8b1-4aa5-8029-05d7d39df876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:36.665486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1004/2009] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:36 2022] DELETE /identity/v3/projects/09f9dd9f8cfe4b9a82d0d21a6bacb132 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:36.670735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b4392-1569-465f-8b1a-b27a1d6ef032 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:36.671243 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b4392-1569-465f-8b1a-b27a1d6ef032 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:36.671337 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1b4392-1569-465f-8b1a-b27a1d6ef032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:36.692937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9a1b4392-1569-465f-8b1a-b27a1d6ef032 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:36.694190 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9a1b4392-1569-465f-8b1a-b27a1d6ef032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:36.724194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:36.756864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1006/2010] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:36.764719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67721345-a190-4e89-924c-894d2249f6bf None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:36.765002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67721345-a190-4e89-924c-894d2249f6bf None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:36.765327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67721345-a190-4e89-924c-894d2249f6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:36.784652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-67721345-a190-4e89-924c-894d2249f6bf None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:36.785880 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-67721345-a190-4e89-924c-894d2249f6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:36.794453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1005/2011] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:36.798635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b4392-1569-465f-8b1a-b27a1d6ef032 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:36.809049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1b4392-1569-465f-8b1a-b27a1d6ef032 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:36.813759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7fd7f-6cc8-4462-adaa-88aee487c638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:36.814385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7fd7f-6cc8-4462-adaa-88aee487c638 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:36.814658 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7fd7f-6cc8-4462-adaa-88aee487c638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:36.814924 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d7fd7f-6cc8-4462-adaa-88aee487c638 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:36.815563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7fd7f-6cc8-4462-adaa-88aee487c638 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:13:36.815859 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7fd7f-6cc8-4462-adaa-88aee487c638 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:36.817475 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d7fd7f-6cc8-4462-adaa-88aee487c638 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:36.828980 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1007/2012] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:13:36.832949 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67721345-a190-4e89-924c-894d2249f6bf None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:36.842071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67721345-a190-4e89-924c-894d2249f6bf None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:36.846697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdfe97-be24-44cc-bb8f-a460973d4e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:36.847281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdfe97-be24-44cc-bb8f-a460973d4e8d admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:36.847557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdfe97-be24-44cc-bb8f-a460973d4e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:36.847828 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fdfe97-be24-44cc-bb8f-a460973d4e8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:36.848454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdfe97-be24-44cc-bb8f-a460973d4e8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:36.850366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fdfe97-be24-44cc-bb8f-a460973d4e8d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:36.878443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1006/2013] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:36 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:13:36.881971 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7fd7f-6cc8-4462-adaa-88aee487c638 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:36.891800 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d7fd7f-6cc8-4462-adaa-88aee487c638 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:36.946904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b57d3d-6d80-40e3-b856-69e0bbacd53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:36.947505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b57d3d-6d80-40e3-b856-69e0bbacd53a admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:36.947786 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b57d3d-6d80-40e3-b856-69e0bbacd53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:36.948059 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b57d3d-6d80-40e3-b856-69e0bbacd53a admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:36.948687 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b57d3d-6d80-40e3-b856-69e0bbacd53a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:36.950613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b57d3d-6d80-40e3-b856-69e0bbacd53a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:36.971429 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1008/2014] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:36 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:36.975621 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdfe97-be24-44cc-bb8f-a460973d4e8d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:36.987491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fdfe97-be24-44cc-bb8f-a460973d4e8d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:36.992190 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c8a20-c1f9-4cbf-b71f-e19c4fe79ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:36.992795 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c8a20-c1f9-4cbf-b71f-e19c4fe79ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:36.993072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c8a20-c1f9-4cbf-b71f-e19c4fe79ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:36.994280 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46c8a20-c1f9-4cbf-b71f-e19c4fe79ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:36.994915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c8a20-c1f9-4cbf-b71f-e19c4fe79ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:36.997367 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46c8a20-c1f9-4cbf-b71f-e19c4fe79ed1 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:37.008013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1007/2015] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:37.012216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b57d3d-6d80-40e3-b856-69e0bbacd53a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:37.022370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b57d3d-6d80-40e3-b856-69e0bbacd53a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:37.027619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e350580-0b83-40f5-bdee-42432779bdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:37.028221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e350580-0b83-40f5-bdee-42432779bdb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.028498 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e350580-0b83-40f5-bdee-42432779bdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.028768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e350580-0b83-40f5-bdee-42432779bdb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.029430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e350580-0b83-40f5-bdee-42432779bdb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:37.031006 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e350580-0b83-40f5-bdee-42432779bdb7 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:37.042413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1009/2016] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:37.046600 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c8a20-c1f9-4cbf-b71f-e19c4fe79ed1 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:37.058653 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46c8a20-c1f9-4cbf-b71f-e19c4fe79ed1 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:37.063433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22f64c-c6be-46f5-bacd-c4bde6702208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:37.064088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22f64c-c6be-46f5-bacd-c4bde6702208 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.064383 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22f64c-c6be-46f5-bacd-c4bde6702208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.064650 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd22f64c-c6be-46f5-bacd-c4bde6702208 admin admin] PATH_INFO: `/v3/projects/ee8d76afdceb42edb3894caeb86e1e65/users/a7a3a466102647e8862f2985a4ed0d6e/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.079415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22f64c-c6be-46f5-bacd-c4bde6702208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee8d76afdceb42edb3894caeb86e1e65, user_id=a7a3a466102647e8862f2985a4ed0d6e, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:37.081969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd22f64c-c6be-46f5-bacd-c4bde6702208 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:37.095652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1008/2017] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:37 2022] PUT /identity/v3/projects/ee8d76afdceb42edb3894caeb86e1e65/users/a7a3a466102647e8862f2985a4ed0d6e/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:37.107448 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117b42e-2e9f-49ab-8277-5131c1739ae4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.108334 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117b42e-2e9f-49ab-8277-5131c1739ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.108608 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1117b42e-2e9f-49ab-8277-5131c1739ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.137421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-1117b42e-2e9f-49ab-8277-5131c1739ae4 None None] MFA Rules not processed for user `a7a3a466102647e8862f2985a4ed0d6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:37.139154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1117b42e-2e9f-49ab-8277-5131c1739ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:37.214165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1010/2018] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:13:37.215679 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22f64c-c6be-46f5-bacd-c4bde6702208 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:37.289210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd22f64c-c6be-46f5-bacd-c4bde6702208 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:37.295375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307c68e-d5ee-4eaf-a7c4-01955a0f96b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:37.295598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307c68e-d5ee-4eaf-a7c4-01955a0f96b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.295833 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307c68e-d5ee-4eaf-a7c4-01955a0f96b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.296101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5307c68e-d5ee-4eaf-a7c4-01955a0f96b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.296720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307c68e-d5ee-4eaf-a7c4-01955a0f96b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:37.298645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5307c68e-d5ee-4eaf-a7c4-01955a0f96b3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:37.323329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:37.331380 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1009/2019] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:37 2022] POST /identity/v3/projects => generated 387 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:37.335295 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117b42e-2e9f-49ab-8277-5131c1739ae4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:37.347514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1117b42e-2e9f-49ab-8277-5131c1739ae4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:37.390669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:37.428153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e300e9-3ccb-4b3a-953d-73111eab84a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:37.428770 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e300e9-3ccb-4b3a-953d-73111eab84a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.429051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e300e9-3ccb-4b3a-953d-73111eab84a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.429354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e300e9-3ccb-4b3a-953d-73111eab84a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.430126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e300e9-3ccb-4b3a-953d-73111eab84a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:37.432090 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e300e9-3ccb-4b3a-953d-73111eab84a6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:37.466036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307c68e-d5ee-4eaf-a7c4-01955a0f96b3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:37.468713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1011/2020] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:37 2022] POST /identity/v3/users => generated 300 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:37.478427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5307c68e-d5ee-4eaf-a7c4-01955a0f96b3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:37.485891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe51bb6-82f1-47b5-a9b8-7a454a2c3fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:37.486476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe51bb6-82f1-47b5-a9b8-7a454a2c3fa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.486954 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe51bb6-82f1-47b5-a9b8-7a454a2c3fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.487266 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe51bb6-82f1-47b5-a9b8-7a454a2c3fa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.487983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe51bb6-82f1-47b5-a9b8-7a454a2c3fa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:37.489754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe51bb6-82f1-47b5-a9b8-7a454a2c3fa4 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:37.504476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1010/2021] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:13:37.508323 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e300e9-3ccb-4b3a-953d-73111eab84a6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:37.519936 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e300e9-3ccb-4b3a-953d-73111eab84a6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:37.532684 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7818e07-4fe7-4a1f-84b4-e66792c16101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:37.533398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7818e07-4fe7-4a1f-84b4-e66792c16101 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.533675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7818e07-4fe7-4a1f-84b4-e66792c16101 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.533947 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7818e07-4fe7-4a1f-84b4-e66792c16101 admin admin] PATH_INFO: `/v3/projects/600e064e7cc444c1a5392f1f2466b56e/users/178e66f98f9b4acd8f5dc63374729f96/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.561556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7818e07-4fe7-4a1f-84b4-e66792c16101 admin admin] RBAC: Authorizing `identity:create_grant(project_id=600e064e7cc444c1a5392f1f2466b56e, user_id=178e66f98f9b4acd8f5dc63374729f96, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:37.564835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7818e07-4fe7-4a1f-84b4-e66792c16101 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:37.584921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1012/2022] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:37 2022] PUT /identity/v3/projects/600e064e7cc444c1a5392f1f2466b56e/users/178e66f98f9b4acd8f5dc63374729f96/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:37.589882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe51bb6-82f1-47b5-a9b8-7a454a2c3fa4 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:37.610462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe51bb6-82f1-47b5-a9b8-7a454a2c3fa4 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:37.681698 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7c4152-6d8a-456a-b2c7-b9f428a07bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:37.682326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c4152-6d8a-456a-b2c7-b9f428a07bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.682613 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c4152-6d8a-456a-b2c7-b9f428a07bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.682876 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7c4152-6d8a-456a-b2c7-b9f428a07bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.683585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c4152-6d8a-456a-b2c7-b9f428a07bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:37.685282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7c4152-6d8a-456a-b2c7-b9f428a07bb4 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:37.695506 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1011/2023] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:37 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:37.700196 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7818e07-4fe7-4a1f-84b4-e66792c16101 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:37.710271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7818e07-4fe7-4a1f-84b4-e66792c16101 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:37.715459 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25408938-a31e-4f76-9349-6888d6fb7726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:37.716094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25408938-a31e-4f76-9349-6888d6fb7726 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.716365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25408938-a31e-4f76-9349-6888d6fb7726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.716631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25408938-a31e-4f76-9349-6888d6fb7726 admin admin] PATH_INFO: `/v3/projects/600e064e7cc444c1a5392f1f2466b56e/users/178e66f98f9b4acd8f5dc63374729f96/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.718540 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25408938-a31e-4f76-9349-6888d6fb7726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=600e064e7cc444c1a5392f1f2466b56e, user_id=178e66f98f9b4acd8f5dc63374729f96, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:37.721092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25408938-a31e-4f76-9349-6888d6fb7726 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:37.730973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1013/2024] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:37 2022] PUT /identity/v3/projects/600e064e7cc444c1a5392f1f2466b56e/users/178e66f98f9b4acd8f5dc63374729f96/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:37.741179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bfc0b1-a73c-4a6f-8632-60128f810517 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.741436 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bfc0b1-a73c-4a6f-8632-60128f810517 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.741710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bfc0b1-a73c-4a6f-8632-60128f810517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.770322 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-81bfc0b1-a73c-4a6f-8632-60128f810517 None None] MFA Rules not processed for user `178e66f98f9b4acd8f5dc63374729f96`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:37.771511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-81bfc0b1-a73c-4a6f-8632-60128f810517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:37.839766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1012/2025] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:37 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:13:37.851032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25408938-a31e-4f76-9349-6888d6fb7726 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:37.852741 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-25408938-a31e-4f76-9349-6888d6fb7726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:37.913439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25408938-a31e-4f76-9349-6888d6fb7726 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:37.917083 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f0f9d-9209-4766-b370-5a74ce1d5a63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:37.917723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f0f9d-9209-4766-b370-5a74ce1d5a63 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:37.918006 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f0f9d-9209-4766-b370-5a74ce1d5a63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:37.918278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f0f9d-9209-4766-b370-5a74ce1d5a63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:37.927511 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f0f9d-9209-4766-b370-5a74ce1d5a63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:37.929060 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f0f9d-9209-4766-b370-5a74ce1d5a63 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:37.939391 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1014/2026] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:37.968246 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bfc0b1-a73c-4a6f-8632-60128f810517 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:38.034149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bfc0b1-a73c-4a6f-8632-60128f810517 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:38.039027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fea295-3a17-49af-9217-cded3115978e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:38.039719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fea295-3a17-49af-9217-cded3115978e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:38.040024 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fea295-3a17-49af-9217-cded3115978e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:38.040293 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fea295-3a17-49af-9217-cded3115978e admin admin] PATH_INFO: `/v3/users/a7a3a466102647e8862f2985a4ed0d6e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:38.041755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fea295-3a17-49af-9217-cded3115978e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a3a466102647e8862f2985a4ed0d6e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:38.043994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fea295-3a17-49af-9217-cded3115978e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:38.058413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:38.096038 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-63fea295-3a17-49af-9217-cded3115978e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a3a466102647e8862f2985a4ed0d6e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:38.103583 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-63fea295-3a17-49af-9217-cded3115978e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a3a466102647e8862f2985a4ed0d6e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:38.116070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-63fea295-3a17-49af-9217-cded3115978e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a3a466102647e8862f2985a4ed0d6e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:38.116797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-63fea295-3a17-49af-9217-cded3115978e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a3a466102647e8862f2985a4ed0d6e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:38.117453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-63fea295-3a17-49af-9217-cded3115978e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a3a466102647e8862f2985a4ed0d6e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:38.131586 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1013/2027] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:37 2022] DELETE /identity/v3/users/a7a3a466102647e8862f2985a4ed0d6e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:38.137660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd020e-6892-4ac7-9abd-331ff2276301 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:38.137962 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd020e-6892-4ac7-9abd-331ff2276301 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:38.138254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfd020e-6892-4ac7-9abd-331ff2276301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:38.159261 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-fcfd020e-6892-4ac7-9abd-331ff2276301 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:38.160462 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-fcfd020e-6892-4ac7-9abd-331ff2276301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:38.226703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1015/2028] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:38.238659 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fea295-3a17-49af-9217-cded3115978e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:38.240557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-63fea295-3a17-49af-9217-cded3115978e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:38.307838 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fea295-3a17-49af-9217-cded3115978e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:38.311515 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ca0e5-5cdb-41e1-8187-85757ccee7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:38.312190 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ca0e5-5cdb-41e1-8187-85757ccee7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:38.312468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ca0e5-5cdb-41e1-8187-85757ccee7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:38.312739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ca0e5-5cdb-41e1-8187-85757ccee7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:38.322430 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ca0e5-5cdb-41e1-8187-85757ccee7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:38.323845 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97ca0e5-5cdb-41e1-8187-85757ccee7a9 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:38.333973 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1014/2029] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:13:38.577227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd020e-6892-4ac7-9abd-331ff2276301 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:38.578757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-fcfd020e-6892-4ac7-9abd-331ff2276301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:38.592596 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfd020e-6892-4ac7-9abd-331ff2276301 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:38.598788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c500e7-a669-4103-ba97-3167af008c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:38.599589 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c500e7-a669-4103-ba97-3167af008c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:38.599976 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c500e7-a669-4103-ba97-3167af008c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:38.600277 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c500e7-a669-4103-ba97-3167af008c53 admin admin] PATH_INFO: `/v3/users/6abc049e0f024d559b2eb82ecaf45959` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:38.601573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c500e7-a669-4103-ba97-3167af008c53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6abc049e0f024d559b2eb82ecaf45959)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:38.603700 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c500e7-a669-4103-ba97-3167af008c53 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:38.636825 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:38.699254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-54c500e7-a669-4103-ba97-3167af008c53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6abc049e0f024d559b2eb82ecaf45959'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:38.716842 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-54c500e7-a669-4103-ba97-3167af008c53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6abc049e0f024d559b2eb82ecaf45959'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:38.731687 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-54c500e7-a669-4103-ba97-3167af008c53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6abc049e0f024d559b2eb82ecaf45959'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:38.732685 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-54c500e7-a669-4103-ba97-3167af008c53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6abc049e0f024d559b2eb82ecaf45959'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:38.750636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-54c500e7-a669-4103-ba97-3167af008c53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6abc049e0f024d559b2eb82ecaf45959'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:38.765600 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1016/2030] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:38 2022] DELETE /identity/v3/users/6abc049e0f024d559b2eb82ecaf45959 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:38.765791 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75851ddf-f2cb-471d-9c11-e743c532a41b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:38.766232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75851ddf-f2cb-471d-9c11-e743c532a41b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:38.766307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75851ddf-f2cb-471d-9c11-e743c532a41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:38.801772 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-75851ddf-f2cb-471d-9c11-e743c532a41b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:38.803015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-75851ddf-f2cb-471d-9c11-e743c532a41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:38.911912 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1015/2031] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:38.930115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c500e7-a669-4103-ba97-3167af008c53 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:38.933073 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-54c500e7-a669-4103-ba97-3167af008c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:39.026019 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c500e7-a669-4103-ba97-3167af008c53 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:39.031105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb442c9-a38e-413d-bd75-401da03b00d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:39.032050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb442c9-a38e-413d-bd75-401da03b00d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:39.032397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb442c9-a38e-413d-bd75-401da03b00d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:39.032673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb442c9-a38e-413d-bd75-401da03b00d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:39.045256 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb442c9-a38e-413d-bd75-401da03b00d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:39.047365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb442c9-a38e-413d-bd75-401da03b00d4 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:39.070587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1017/2032] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:39.117555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75851ddf-f2cb-471d-9c11-e743c532a41b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:39.119135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-75851ddf-f2cb-471d-9c11-e743c532a41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:39.155982 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75851ddf-f2cb-471d-9c11-e743c532a41b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:39.163763 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d213393-b549-4d8d-9b4f-77fe1eb427a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:39.165009 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d213393-b549-4d8d-9b4f-77fe1eb427a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:39.165330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d213393-b549-4d8d-9b4f-77fe1eb427a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:39.165608 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d213393-b549-4d8d-9b4f-77fe1eb427a8 admin admin] PATH_INFO: `/v3/projects/ee8d76afdceb42edb3894caeb86e1e65` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:39.167141 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d213393-b549-4d8d-9b4f-77fe1eb427a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee8d76afdceb42edb3894caeb86e1e65)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:39.169851 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d213393-b549-4d8d-9b4f-77fe1eb427a8 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:39.232427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:39.258125 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3d213393-b549-4d8d-9b4f-77fe1eb427a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee8d76afdceb42edb3894caeb86e1e65', 'request_id': 'req-3d213393-b549-4d8d-9b4f-77fe1eb427a8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.278113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3d213393-b549-4d8d-9b4f-77fe1eb427a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee8d76afdceb42edb3894caeb86e1e65', 'request_id': 'req-3d213393-b549-4d8d-9b4f-77fe1eb427a8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.284905 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3d213393-b549-4d8d-9b4f-77fe1eb427a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee8d76afdceb42edb3894caeb86e1e65', 'request_id': 'req-3d213393-b549-4d8d-9b4f-77fe1eb427a8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.298848 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3d213393-b549-4d8d-9b4f-77fe1eb427a8 admin admin] The token cache is being invalidate because project ee8d76afdceb42edb3894caeb86e1e65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:39.299411 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3d213393-b549-4d8d-9b4f-77fe1eb427a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.300307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3d213393-b549-4d8d-9b4f-77fe1eb427a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.302171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1016/2033] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:39 2022] DELETE /identity/v3/projects/ee8d76afdceb42edb3894caeb86e1e65 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:39.307059 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb442c9-a38e-413d-bd75-401da03b00d4 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:39.323834 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb442c9-a38e-413d-bd75-401da03b00d4 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:39.411869 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e13518-0ad0-4c7a-99e8-1e25fb6f7bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:39.412534 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e13518-0ad0-4c7a-99e8-1e25fb6f7bb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:39.412900 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e13518-0ad0-4c7a-99e8-1e25fb6f7bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:39.413089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e13518-0ad0-4c7a-99e8-1e25fb6f7bb6 admin admin] PATH_INFO: `/v3/users/178e66f98f9b4acd8f5dc63374729f96` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:39.414694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e13518-0ad0-4c7a-99e8-1e25fb6f7bb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=178e66f98f9b4acd8f5dc63374729f96)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:39.416933 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e13518-0ad0-4c7a-99e8-1e25fb6f7bb6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:39.473537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:39.479943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c5e13518-0ad0-4c7a-99e8-1e25fb6f7bb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '178e66f98f9b4acd8f5dc63374729f96'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.492451 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c5e13518-0ad0-4c7a-99e8-1e25fb6f7bb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '178e66f98f9b4acd8f5dc63374729f96'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.503590 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c5e13518-0ad0-4c7a-99e8-1e25fb6f7bb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '178e66f98f9b4acd8f5dc63374729f96'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.504212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c5e13518-0ad0-4c7a-99e8-1e25fb6f7bb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '178e66f98f9b4acd8f5dc63374729f96'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.521328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c5e13518-0ad0-4c7a-99e8-1e25fb6f7bb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '178e66f98f9b4acd8f5dc63374729f96'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.522198 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d213393-b549-4d8d-9b4f-77fe1eb427a8 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:39.524498 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3d213393-b549-4d8d-9b4f-77fe1eb427a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:39.535274 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1018/2034] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:39 2022] DELETE /identity/v3/users/178e66f98f9b4acd8f5dc63374729f96 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:39.550104 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d213393-b549-4d8d-9b4f-77fe1eb427a8 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:39.640391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0c9d7-5150-46d8-b9ed-4887890450f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:39.640391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0c9d7-5150-46d8-b9ed-4887890450f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:39.640391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0c9d7-5150-46d8-b9ed-4887890450f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:39.640391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b0c9d7-5150-46d8-b9ed-4887890450f4 admin admin] PATH_INFO: `/v3/projects/2250e2ff2c524c328865f1752dc98a6d` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:39.640719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0c9d7-5150-46d8-b9ed-4887890450f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2250e2ff2c524c328865f1752dc98a6d)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:39.641585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b0c9d7-5150-46d8-b9ed-4887890450f4 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:39.726149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-90b0c9d7-5150-46d8-b9ed-4887890450f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2250e2ff2c524c328865f1752dc98a6d', 'request_id': 'req-90b0c9d7-5150-46d8-b9ed-4887890450f4'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.740518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-90b0c9d7-5150-46d8-b9ed-4887890450f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2250e2ff2c524c328865f1752dc98a6d', 'request_id': 'req-90b0c9d7-5150-46d8-b9ed-4887890450f4'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.749035 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-90b0c9d7-5150-46d8-b9ed-4887890450f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2250e2ff2c524c328865f1752dc98a6d', 'request_id': 'req-90b0c9d7-5150-46d8-b9ed-4887890450f4'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.767917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-90b0c9d7-5150-46d8-b9ed-4887890450f4 admin admin] The token cache is being invalidate because project 2250e2ff2c524c328865f1752dc98a6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:39.768537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-90b0c9d7-5150-46d8-b9ed-4887890450f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.768862 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-90b0c9d7-5150-46d8-b9ed-4887890450f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.773251 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1017/2035] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:39 2022] DELETE /identity/v3/projects/2250e2ff2c524c328865f1752dc98a6d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:39.774502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e13518-0ad0-4c7a-99e8-1e25fb6f7bb6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:39.776442 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c5e13518-0ad0-4c7a-99e8-1e25fb6f7bb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:39.813415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e13518-0ad0-4c7a-99e8-1e25fb6f7bb6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:39.888536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6be8c1a-0c9b-4e80-b9fb-b31160515ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:39.889103 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be8c1a-0c9b-4e80-b9fb-b31160515ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:39.889474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be8c1a-0c9b-4e80-b9fb-b31160515ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:39.889854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6be8c1a-0c9b-4e80-b9fb-b31160515ea4 admin admin] PATH_INFO: `/v3/users/7b427a3e4fa04eef9556ec83e76f8595` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:39.891522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be8c1a-0c9b-4e80-b9fb-b31160515ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b427a3e4fa04eef9556ec83e76f8595)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:39.895223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6be8c1a-0c9b-4e80-b9fb-b31160515ea4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:39.965123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e6be8c1a-0c9b-4e80-b9fb-b31160515ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b427a3e4fa04eef9556ec83e76f8595'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.972916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e6be8c1a-0c9b-4e80-b9fb-b31160515ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b427a3e4fa04eef9556ec83e76f8595'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.983120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e6be8c1a-0c9b-4e80-b9fb-b31160515ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b427a3e4fa04eef9556ec83e76f8595'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:39.983831 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e6be8c1a-0c9b-4e80-b9fb-b31160515ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b427a3e4fa04eef9556ec83e76f8595'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.009979 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e6be8c1a-0c9b-4e80-b9fb-b31160515ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b427a3e4fa04eef9556ec83e76f8595'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.018833 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1019/2036] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:39 2022] DELETE /identity/v3/users/7b427a3e4fa04eef9556ec83e76f8595 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:40.130281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0c9d7-5150-46d8-b9ed-4887890450f4 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:40.132358 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-90b0c9d7-5150-46d8-b9ed-4887890450f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:40.159298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b0c9d7-5150-46d8-b9ed-4887890450f4 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:40.254760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:40.278477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941146e-5c64-4b04-80b8-fa73788b1767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:40.279613 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941146e-5c64-4b04-80b8-fa73788b1767 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:40.280238 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941146e-5c64-4b04-80b8-fa73788b1767 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:40.280576 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5941146e-5c64-4b04-80b8-fa73788b1767 admin admin] PATH_INFO: `/v3/projects/600e064e7cc444c1a5392f1f2466b56e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:40.282072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941146e-5c64-4b04-80b8-fa73788b1767 admin admin] RBAC: Authorizing `identity:delete_project(project_id=600e064e7cc444c1a5392f1f2466b56e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:40.284116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5941146e-5c64-4b04-80b8-fa73788b1767 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:40.378923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5941146e-5c64-4b04-80b8-fa73788b1767 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '600e064e7cc444c1a5392f1f2466b56e', 'request_id': 'req-5941146e-5c64-4b04-80b8-fa73788b1767'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.397420 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5941146e-5c64-4b04-80b8-fa73788b1767 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '600e064e7cc444c1a5392f1f2466b56e', 'request_id': 'req-5941146e-5c64-4b04-80b8-fa73788b1767'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.423993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5941146e-5c64-4b04-80b8-fa73788b1767 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '600e064e7cc444c1a5392f1f2466b56e', 'request_id': 'req-5941146e-5c64-4b04-80b8-fa73788b1767'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.438930 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5941146e-5c64-4b04-80b8-fa73788b1767 admin admin] The token cache is being invalidate because project 600e064e7cc444c1a5392f1f2466b56e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:40.439231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5941146e-5c64-4b04-80b8-fa73788b1767 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.440005 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5941146e-5c64-4b04-80b8-fa73788b1767 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.442157 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1018/2037] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:40 2022] DELETE /identity/v3/projects/600e064e7cc444c1a5392f1f2466b56e => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:13:40.448324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1928e-0be4-4136-9e52-61b8cb450157 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:40.448614 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1928e-0be4-4136-9e52-61b8cb450157 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:40.448892 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1928e-0be4-4136-9e52-61b8cb450157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:40.471109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c3a1928e-0be4-4136-9e52-61b8cb450157 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:40.472445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c3a1928e-0be4-4136-9e52-61b8cb450157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:40.485134 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941146e-5c64-4b04-80b8-fa73788b1767 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:40.486862 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5941146e-5c64-4b04-80b8-fa73788b1767 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:40.541886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:40.591198 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5941146e-5c64-4b04-80b8-fa73788b1767 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:40.593526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1020/2038] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:40.601825 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965f5d1-512f-4c3d-a59d-67f659f17e79 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:40.602281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965f5d1-512f-4c3d-a59d-67f659f17e79 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:40.602776 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7965f5d1-512f-4c3d-a59d-67f659f17e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:40.603606 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e84cdde-a1ee-4a89-9373-25488268e74c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:40.604714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84cdde-a1ee-4a89-9373-25488268e74c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:40.605224 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84cdde-a1ee-4a89-9373-25488268e74c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:40.605869 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e84cdde-a1ee-4a89-9373-25488268e74c admin admin] PATH_INFO: `/v3/projects/66319069c1e647cc9f193e9b6e8d10bd` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:40.607085 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84cdde-a1ee-4a89-9373-25488268e74c admin admin] RBAC: Authorizing `identity:delete_project(project_id=66319069c1e647cc9f193e9b6e8d10bd)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:40.609646 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e84cdde-a1ee-4a89-9373-25488268e74c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:40.627389 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7965f5d1-512f-4c3d-a59d-67f659f17e79 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:40.628873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7965f5d1-512f-4c3d-a59d-67f659f17e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:40.642515 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1021/2039] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:13:40.647733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965f5d1-512f-4c3d-a59d-67f659f17e79 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:40.661857 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7965f5d1-512f-4c3d-a59d-67f659f17e79 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:40.670145 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d6697-e673-4282-a1e8-1a56396bc668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:40.671053 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d6697-e673-4282-a1e8-1a56396bc668 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:40.672780 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d6697-e673-4282-a1e8-1a56396bc668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:40.673413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9d6697-e673-4282-a1e8-1a56396bc668 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:40.674172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d6697-e673-4282-a1e8-1a56396bc668 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:13:40.674578 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d6697-e673-4282-a1e8-1a56396bc668 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:40.676298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9d6697-e673-4282-a1e8-1a56396bc668 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:40.688213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4e84cdde-a1ee-4a89-9373-25488268e74c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66319069c1e647cc9f193e9b6e8d10bd', 'request_id': 'req-4e84cdde-a1ee-4a89-9373-25488268e74c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.693023 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1022/2040] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:13:40.697517 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d6697-e673-4282-a1e8-1a56396bc668 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:40.698839 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4e84cdde-a1ee-4a89-9373-25488268e74c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66319069c1e647cc9f193e9b6e8d10bd', 'request_id': 'req-4e84cdde-a1ee-4a89-9373-25488268e74c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.707764 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4e84cdde-a1ee-4a89-9373-25488268e74c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66319069c1e647cc9f193e9b6e8d10bd', 'request_id': 'req-4e84cdde-a1ee-4a89-9373-25488268e74c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.710210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9d6697-e673-4282-a1e8-1a56396bc668 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:40.725974 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4e84cdde-a1ee-4a89-9373-25488268e74c admin admin] The token cache is being invalidate because project 66319069c1e647cc9f193e9b6e8d10bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:40.727237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4e84cdde-a1ee-4a89-9373-25488268e74c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.728608 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-4e84cdde-a1ee-4a89-9373-25488268e74c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.735062 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1019/2041] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:40 2022] DELETE /identity/v3/projects/66319069c1e647cc9f193e9b6e8d10bd => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:13:40.737362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dca1a-f93b-415f-aa43-793c95f9e328 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:40.737645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dca1a-f93b-415f-aa43-793c95f9e328 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:40.737915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35dca1a-f93b-415f-aa43-793c95f9e328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:40.753602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:40.761451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a35dca1a-f93b-415f-aa43-793c95f9e328 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:40.763456 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a35dca1a-f93b-415f-aa43-793c95f9e328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:40.804778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8982e1a-af19-4c4e-85e7-205a92ee9147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:40.805484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8982e1a-af19-4c4e-85e7-205a92ee9147 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:40.806796 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8982e1a-af19-4c4e-85e7-205a92ee9147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:40.807069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8982e1a-af19-4c4e-85e7-205a92ee9147 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:40.807696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8982e1a-af19-4c4e-85e7-205a92ee9147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:40.809682 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8982e1a-af19-4c4e-85e7-205a92ee9147 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:40.830355 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1020/2042] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:13:40.831692 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35dca1a-f93b-415f-aa43-793c95f9e328 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:40.832474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a35dca1a-f93b-415f-aa43-793c95f9e328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:40.846303 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35dca1a-f93b-415f-aa43-793c95f9e328 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:40.852031 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1023/2043] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:40 2022] POST /identity/v3/projects => generated 383 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:40.852031 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1298a-83ee-4955-a5f3-d857984a64fc None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:40.852031 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1298a-83ee-4955-a5f3-d857984a64fc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:40.852031 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f1298a-83ee-4955-a5f3-d857984a64fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:40.855366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfc953-809f-4872-89b4-a1349efbc27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:40.856006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfc953-809f-4872-89b4-a1349efbc27f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:40.856276 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfc953-809f-4872-89b4-a1349efbc27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:40.856544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cfc953-809f-4872-89b4-a1349efbc27f admin admin] PATH_INFO: `/v3/users/919e0a47fcb745229a7d3c7d1f10dc8e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:40.858120 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfc953-809f-4872-89b4-a1349efbc27f admin admin] RBAC: Authorizing `identity:delete_user(user_id=919e0a47fcb745229a7d3c7d1f10dc8e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:40.860179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81cfc953-809f-4872-89b4-a1349efbc27f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:40.874235 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c9f1298a-83ee-4955-a5f3-d857984a64fc None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:40.875464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c9f1298a-83ee-4955-a5f3-d857984a64fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:40.914440 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-81cfc953-809f-4872-89b4-a1349efbc27f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '919e0a47fcb745229a7d3c7d1f10dc8e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.922708 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-81cfc953-809f-4872-89b4-a1349efbc27f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '919e0a47fcb745229a7d3c7d1f10dc8e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.937040 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-81cfc953-809f-4872-89b4-a1349efbc27f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '919e0a47fcb745229a7d3c7d1f10dc8e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.938223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-81cfc953-809f-4872-89b4-a1349efbc27f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '919e0a47fcb745229a7d3c7d1f10dc8e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.938546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-81cfc953-809f-4872-89b4-a1349efbc27f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '919e0a47fcb745229a7d3c7d1f10dc8e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:40.941012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1024/2044] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:40.941790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1298a-83ee-4955-a5f3-d857984a64fc None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:40.943236 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c9f1298a-83ee-4955-a5f3-d857984a64fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:40.956042 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f1298a-83ee-4955-a5f3-d857984a64fc None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:40.956707 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1021/2045] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:40 2022] DELETE /identity/v3/users/919e0a47fcb745229a7d3c7d1f10dc8e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:40.961554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfc953-809f-4872-89b4-a1349efbc27f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:40.963167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-81cfc953-809f-4872-89b4-a1349efbc27f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:41.002133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:41.022685 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e81bdf-22de-431b-b958-b65e02b766f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:41.023395 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e81bdf-22de-431b-b958-b65e02b766f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.023748 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e81bdf-22de-431b-b958-b65e02b766f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.024716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e81bdf-22de-431b-b958-b65e02b766f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.025365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e81bdf-22de-431b-b958-b65e02b766f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:41.027462 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e81bdf-22de-431b-b958-b65e02b766f7 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:41.037146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cfc953-809f-4872-89b4-a1349efbc27f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:41.043535 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e9e23-7b77-4f53-8da3-ddbc995f602c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:41.044279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e9e23-7b77-4f53-8da3-ddbc995f602c admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.044651 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e9e23-7b77-4f53-8da3-ddbc995f602c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.045071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16e9e23-7b77-4f53-8da3-ddbc995f602c admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.046014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e9e23-7b77-4f53-8da3-ddbc995f602c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:13:41.046406 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e9e23-7b77-4f53-8da3-ddbc995f602c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:41.048136 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16e9e23-7b77-4f53-8da3-ddbc995f602c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:41.051137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1025/2046] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:40 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:41.052604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b040e4-9ebd-42c8-b84f-41ce78159d9b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.052990 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b040e4-9ebd-42c8-b84f-41ce78159d9b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.053948 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b040e4-9ebd-42c8-b84f-41ce78159d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.069641 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1022/2047] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:13:41.070618 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e9e23-7b77-4f53-8da3-ddbc995f602c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:41.072369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c16e9e23-7b77-4f53-8da3-ddbc995f602c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:41.075570 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a2b040e4-9ebd-42c8-b84f-41ce78159d9b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:41.076861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a2b040e4-9ebd-42c8-b84f-41ce78159d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:41.088560 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1026/2048] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:41.090776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16e9e23-7b77-4f53-8da3-ddbc995f602c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:41.091301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b040e4-9ebd-42c8-b84f-41ce78159d9b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:41.103077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b0371-868d-4d9d-bd67-9baad85a9289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:41.103587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b0371-868d-4d9d-bd67-9baad85a9289 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.103862 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b0371-868d-4d9d-bd67-9baad85a9289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.104131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b0371-868d-4d9d-bd67-9baad85a9289 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.104769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b0371-868d-4d9d-bd67-9baad85a9289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:41.105462 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b040e4-9ebd-42c8-b84f-41ce78159d9b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:41.106457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478b0371-868d-4d9d-bd67-9baad85a9289 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:41.110714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6d992-2255-44c6-b5ea-933d06a830d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:41.111323 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6d992-2255-44c6-b5ea-933d06a830d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.111605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6d992-2255-44c6-b5ea-933d06a830d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.111884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6d992-2255-44c6-b5ea-933d06a830d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.112574 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6d992-2255-44c6-b5ea-933d06a830d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:41.115530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f6d992-2255-44c6-b5ea-933d06a830d6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:41.124087 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1023/2049] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:41.125403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b0371-868d-4d9d-bd67-9baad85a9289 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:41.128927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-478b0371-868d-4d9d-bd67-9baad85a9289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:41.144424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:41.179372 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1027/2050] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:41 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:41.180902 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6d992-2255-44c6-b5ea-933d06a830d6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:41.257218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b0371-868d-4d9d-bd67-9baad85a9289 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:41.297132 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f6d992-2255-44c6-b5ea-933d06a830d6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:41.305296 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ab46f-9135-4a4b-b383-99905e3960ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:41.306393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ab46f-9135-4a4b-b383-99905e3960ea admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.306826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ab46f-9135-4a4b-b383-99905e3960ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.307357 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55ab46f-9135-4a4b-b383-99905e3960ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.307938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ab46f-9135-4a4b-b383-99905e3960ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:41.309674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55ab46f-9135-4a4b-b383-99905e3960ea admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:41.322160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1028/2051] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:41.323220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ab46f-9135-4a4b-b383-99905e3960ea admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:41.337687 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55ab46f-9135-4a4b-b383-99905e3960ea admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:41.340042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de4c90-b04a-4a4e-ab25-54aae6ab353a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:41.340679 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de4c90-b04a-4a4e-ab25-54aae6ab353a service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.340963 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de4c90-b04a-4a4e-ab25-54aae6ab353a service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.341438 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de4c90-b04a-4a4e-ab25-54aae6ab353a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.344345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6783db5-e307-4844-b2a8-ff9d9bd152c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:41.344963 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6783db5-e307-4844-b2a8-ff9d9bd152c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.345307 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6783db5-e307-4844-b2a8-ff9d9bd152c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.345589 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6783db5-e307-4844-b2a8-ff9d9bd152c7 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.346317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6783db5-e307-4844-b2a8-ff9d9bd152c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:41.348520 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6783db5-e307-4844-b2a8-ff9d9bd152c7 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:41.352425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de4c90-b04a-4a4e-ab25-54aae6ab353a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:41.353975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de4c90-b04a-4a4e-ab25-54aae6ab353a service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:41.367444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1024/2052] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:41.368521 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de4c90-b04a-4a4e-ab25-54aae6ab353a service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:41.380674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1029/2053] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:41 2022] POST /identity/v3/users => generated 287 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:41.380883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de4c90-b04a-4a4e-ab25-54aae6ab353a service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:41.382384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6783db5-e307-4844-b2a8-ff9d9bd152c7 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:41.386443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2984e41-482d-4983-8009-e553a41d549e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:41.387242 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2984e41-482d-4983-8009-e553a41d549e admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.387548 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2984e41-482d-4983-8009-e553a41d549e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.387824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2984e41-482d-4983-8009-e553a41d549e admin admin] PATH_INFO: `/v3/projects/05a68c956c924e8191b44aa24377a725/users/9423f778e1db4fd8af9a3ff5ad0f49c0/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.396587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6783db5-e307-4844-b2a8-ff9d9bd152c7 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:41.403299 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a450c4-cce0-430b-b5e2-fd960e88f733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:41.404182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a450c4-cce0-430b-b5e2-fd960e88f733 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.404555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a450c4-cce0-430b-b5e2-fd960e88f733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.405007 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a450c4-cce0-430b-b5e2-fd960e88f733 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.405801 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a450c4-cce0-430b-b5e2-fd960e88f733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:41.407535 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a450c4-cce0-430b-b5e2-fd960e88f733 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:41.408398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2984e41-482d-4983-8009-e553a41d549e admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a68c956c924e8191b44aa24377a725, user_id=9423f778e1db4fd8af9a3ff5ad0f49c0, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:41.411821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2984e41-482d-4983-8009-e553a41d549e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:41.422500 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1030/2054] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:41.426977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a450c4-cce0-430b-b5e2-fd960e88f733 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:41.434825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1025/2055] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:41 2022] PUT /identity/v3/projects/05a68c956c924e8191b44aa24377a725/users/9423f778e1db4fd8af9a3ff5ad0f49c0/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:41.443686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a450c4-cce0-430b-b5e2-fd960e88f733 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:41.447409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febfaf7-111a-43df-88c0-33d464a6ed6f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.447809 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febfaf7-111a-43df-88c0-33d464a6ed6f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.448086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2febfaf7-111a-43df-88c0-33d464a6ed6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.487141 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2febfaf7-111a-43df-88c0-33d464a6ed6f None None] MFA Rules not processed for user `9423f778e1db4fd8af9a3ff5ad0f49c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:41.488400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2febfaf7-111a-43df-88c0-33d464a6ed6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:41.570931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c68407-7b3e-4ef4-971a-70d190d3c493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:41.572602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c68407-7b3e-4ef4-971a-70d190d3c493 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.573789 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c68407-7b3e-4ef4-971a-70d190d3c493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.574537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c68407-7b3e-4ef4-971a-70d190d3c493 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.575690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c68407-7b3e-4ef4-971a-70d190d3c493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:41.577847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c68407-7b3e-4ef4-971a-70d190d3c493 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:41.581699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1026/2056] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:41.583866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febfaf7-111a-43df-88c0-33d464a6ed6f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:41.597302 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2febfaf7-111a-43df-88c0-33d464a6ed6f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:41.600941 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1031/2057] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:13:41.602593 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c68407-7b3e-4ef4-971a-70d190d3c493 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:41.605493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12524ddd-9740-41bc-9d58-6d40df3e2241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:41.606250 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12524ddd-9740-41bc-9d58-6d40df3e2241 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.606604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12524ddd-9740-41bc-9d58-6d40df3e2241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.607132 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12524ddd-9740-41bc-9d58-6d40df3e2241 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.608097 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12524ddd-9740-41bc-9d58-6d40df3e2241 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:41.610366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12524ddd-9740-41bc-9d58-6d40df3e2241 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:41.619604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c68407-7b3e-4ef4-971a-70d190d3c493 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:41.626763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1466e1-10fd-475c-a24f-7083f1f343ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:41.627954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1466e1-10fd-475c-a24f-7083f1f343ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.628239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1466e1-10fd-475c-a24f-7083f1f343ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.628516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1466e1-10fd-475c-a24f-7083f1f343ca admin admin] PATH_INFO: `/v3/projects/fa54ff36fd244c5bbb42a6db95f92d1b/users/9566c18a8de14ce1923839669c211a8e/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.633095 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:41.652708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1466e1-10fd-475c-a24f-7083f1f343ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa54ff36fd244c5bbb42a6db95f92d1b, user_id=9566c18a8de14ce1923839669c211a8e, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:41.655714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1466e1-10fd-475c-a24f-7083f1f343ca admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:41.656474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1027/2058] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:41 2022] POST /identity/v3/projects => generated 381 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:41.661336 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12524ddd-9740-41bc-9d58-6d40df3e2241 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:41.669463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:41.674628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1032/2059] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:41 2022] PUT /identity/v3/projects/fa54ff36fd244c5bbb42a6db95f92d1b/users/9566c18a8de14ce1923839669c211a8e/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:13:41.690507 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33570f8e-96a2-4025-9a72-c7bc9942623e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.690790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33570f8e-96a2-4025-9a72-c7bc9942623e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.691066 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33570f8e-96a2-4025-9a72-c7bc9942623e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.721927 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-33570f8e-96a2-4025-9a72-c7bc9942623e None None] MFA Rules not processed for user `9566c18a8de14ce1923839669c211a8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:41.723212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-33570f8e-96a2-4025-9a72-c7bc9942623e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:41.774318 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12524ddd-9740-41bc-9d58-6d40df3e2241 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:41.826714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1033/2060] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:13:41.827908 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33570f8e-96a2-4025-9a72-c7bc9942623e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:41.833924 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-33570f8e-96a2-4025-9a72-c7bc9942623e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:41.850612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f553d-81fa-4c09-ba6c-234fc731dea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:41.851151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f553d-81fa-4c09-ba6c-234fc731dea9 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.851434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f553d-81fa-4c09-ba6c-234fc731dea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.851704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f553d-81fa-4c09-ba6c-234fc731dea9 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.852464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1f553d-81fa-4c09-ba6c-234fc731dea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:41.856061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1f553d-81fa-4c09-ba6c-234fc731dea9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:41.889582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1028/2061] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:41 2022] POST /identity/v3/users => generated 297 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:41.892978 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f553d-81fa-4c09-ba6c-234fc731dea9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:41.905254 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f553d-81fa-4c09-ba6c-234fc731dea9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:41.917869 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0227b-65b4-4c92-935d-b5021e63bb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:41.918608 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0227b-65b4-4c92-935d-b5021e63bb12 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.918889 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0227b-65b4-4c92-935d-b5021e63bb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.919157 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0227b-65b4-4c92-935d-b5021e63bb12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.919797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed0227b-65b4-4c92-935d-b5021e63bb12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:41.922107 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed0227b-65b4-4c92-935d-b5021e63bb12 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:41.955168 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33570f8e-96a2-4025-9a72-c7bc9942623e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:41.958681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1029/2062] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:41 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:41.959609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0227b-65b4-4c92-935d-b5021e63bb12 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:41.965029 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c21dc1-36a6-425a-9acb-4985b2492078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:41.965773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c21dc1-36a6-425a-9acb-4985b2492078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:41.966048 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c21dc1-36a6-425a-9acb-4985b2492078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:41.966323 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c21dc1-36a6-425a-9acb-4985b2492078 admin admin] PATH_INFO: `/v3/projects/fee4abdf79dd40c5b7df53a0dbb6cc0f` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:41.967691 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c21dc1-36a6-425a-9acb-4985b2492078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fee4abdf79dd40c5b7df53a0dbb6cc0f)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:41.971017 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c21dc1-36a6-425a-9acb-4985b2492078 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:42.035978 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-30c21dc1-36a6-425a-9acb-4985b2492078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fee4abdf79dd40c5b7df53a0dbb6cc0f', 'request_id': 'req-30c21dc1-36a6-425a-9acb-4985b2492078'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:42.041755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed0227b-65b4-4c92-935d-b5021e63bb12 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:42.046164 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-30c21dc1-36a6-425a-9acb-4985b2492078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fee4abdf79dd40c5b7df53a0dbb6cc0f', 'request_id': 'req-30c21dc1-36a6-425a-9acb-4985b2492078'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:42.054983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-30c21dc1-36a6-425a-9acb-4985b2492078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fee4abdf79dd40c5b7df53a0dbb6cc0f', 'request_id': 'req-30c21dc1-36a6-425a-9acb-4985b2492078'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:42.068726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:42.072655 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-30c21dc1-36a6-425a-9acb-4985b2492078 admin admin] The token cache is being invalidate because project fee4abdf79dd40c5b7df53a0dbb6cc0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:42.073267 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-30c21dc1-36a6-425a-9acb-4985b2492078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:42.074067 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-30c21dc1-36a6-425a-9acb-4985b2492078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:42.077301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1034/2063] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:41 2022] DELETE /identity/v3/projects/fee4abdf79dd40c5b7df53a0dbb6cc0f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:42.077301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c21dc1-36a6-425a-9acb-4985b2492078 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:42.080677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-30c21dc1-36a6-425a-9acb-4985b2492078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:42.087905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:42.140426 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86459a18-ec7b-4b32-86db-8fac3ac32f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:42.141164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86459a18-ec7b-4b32-86db-8fac3ac32f1d admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:42.141657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86459a18-ec7b-4b32-86db-8fac3ac32f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:42.142077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86459a18-ec7b-4b32-86db-8fac3ac32f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:42.142848 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86459a18-ec7b-4b32-86db-8fac3ac32f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:42.144931 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86459a18-ec7b-4b32-86db-8fac3ac32f1d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:42.172806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1030/2064] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:42.175380 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86459a18-ec7b-4b32-86db-8fac3ac32f1d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:42.179372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-86459a18-ec7b-4b32-86db-8fac3ac32f1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:42.198015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c21dc1-36a6-425a-9acb-4985b2492078 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:42.218595 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86459a18-ec7b-4b32-86db-8fac3ac32f1d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:42.220819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d667bd0-fde5-4ae2-840a-ed2f6e34a8c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:42.221387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d667bd0-fde5-4ae2-840a-ed2f6e34a8c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:42.221663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d667bd0-fde5-4ae2-840a-ed2f6e34a8c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:42.221950 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d667bd0-fde5-4ae2-840a-ed2f6e34a8c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:42.222579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d667bd0-fde5-4ae2-840a-ed2f6e34a8c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:42.224012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85f738-8b79-4a52-b776-79ec8b96b8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:42.224472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d667bd0-fde5-4ae2-840a-ed2f6e34a8c9 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:42.224699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85f738-8b79-4a52-b776-79ec8b96b8b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:42.224980 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85f738-8b79-4a52-b776-79ec8b96b8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:42.225279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac85f738-8b79-4a52-b776-79ec8b96b8b9 admin admin] PATH_INFO: `/v3/users/fe93b54595b84cb08687844f9fb38ec8` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:42.226453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85f738-8b79-4a52-b776-79ec8b96b8b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe93b54595b84cb08687844f9fb38ec8)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:42.229571 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac85f738-8b79-4a52-b776-79ec8b96b8b9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:42.259340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1035/2065] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:42 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:42.260303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d667bd0-fde5-4ae2-840a-ed2f6e34a8c9 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:42.262021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9d667bd0-fde5-4ae2-840a-ed2f6e34a8c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:42.282245 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d667bd0-fde5-4ae2-840a-ed2f6e34a8c9 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:42.289556 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ac85f738-8b79-4a52-b776-79ec8b96b8b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe93b54595b84cb08687844f9fb38ec8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:42.292592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed53e1-5c12-4c68-8926-65de43912a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:42.295216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed53e1-5c12-4c68-8926-65de43912a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:42.295623 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed53e1-5c12-4c68-8926-65de43912a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:42.295909 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed53e1-5c12-4c68-8926-65de43912a0b admin admin] PATH_INFO: `/v3/projects/9652c5e369af406691c19d84f92b9011/users/70c06ed9b31e43b48f6703397d67d100/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:42.300233 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ac85f738-8b79-4a52-b776-79ec8b96b8b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe93b54595b84cb08687844f9fb38ec8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:42.327436 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed53e1-5c12-4c68-8926-65de43912a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9652c5e369af406691c19d84f92b9011, user_id=70c06ed9b31e43b48f6703397d67d100, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:42.330674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed53e1-5c12-4c68-8926-65de43912a0b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:42.333683 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ac85f738-8b79-4a52-b776-79ec8b96b8b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe93b54595b84cb08687844f9fb38ec8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:42.334437 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ac85f738-8b79-4a52-b776-79ec8b96b8b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe93b54595b84cb08687844f9fb38ec8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:42.336276 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ac85f738-8b79-4a52-b776-79ec8b96b8b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe93b54595b84cb08687844f9fb38ec8'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:42.353592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1036/2066] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:42 2022] PUT /identity/v3/projects/9652c5e369af406691c19d84f92b9011/users/70c06ed9b31e43b48f6703397d67d100/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:42.354568 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed53e1-5c12-4c68-8926-65de43912a0b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:42.356600 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-12ed53e1-5c12-4c68-8926-65de43912a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:42.360047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1031/2067] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:42 2022] DELETE /identity/v3/users/fe93b54595b84cb08687844f9fb38ec8 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:42.361289 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85f738-8b79-4a52-b776-79ec8b96b8b9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:42.363470 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ac85f738-8b79-4a52-b776-79ec8b96b8b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:42.427312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed53e1-5c12-4c68-8926-65de43912a0b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:42.495016 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de02932-e202-468a-a7ee-7dd8ae0f98b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:42.497824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de02932-e202-468a-a7ee-7dd8ae0f98b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:42.498417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de02932-e202-468a-a7ee-7dd8ae0f98b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:42.498417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de02932-e202-468a-a7ee-7dd8ae0f98b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:42.499132 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de02932-e202-468a-a7ee-7dd8ae0f98b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:42.502348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de02932-e202-468a-a7ee-7dd8ae0f98b7 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:42.514973 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac85f738-8b79-4a52-b776-79ec8b96b8b9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:42.522754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b921f7-99bf-4c52-865c-aefde197dc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:42.525010 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b921f7-99bf-4c52-865c-aefde197dc13 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:42.525438 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b921f7-99bf-4c52-865c-aefde197dc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:42.525845 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b921f7-99bf-4c52-865c-aefde197dc13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:42.527249 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b921f7-99bf-4c52-865c-aefde197dc13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:42.529720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b921f7-99bf-4c52-865c-aefde197dc13 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:42.547050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1032/2068] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:42.547659 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1037/2069] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:42.554889 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de02932-e202-468a-a7ee-7dd8ae0f98b7 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:42.555024 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b921f7-99bf-4c52-865c-aefde197dc13 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:42.576609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de02932-e202-468a-a7ee-7dd8ae0f98b7 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:42.581330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b921f7-99bf-4c52-865c-aefde197dc13 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:42.590949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55268d-198b-4d10-8942-9afa56c343f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:42.591654 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55268d-198b-4d10-8942-9afa56c343f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:42.592164 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55268d-198b-4d10-8942-9afa56c343f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:42.592495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd55268d-198b-4d10-8942-9afa56c343f8 admin admin] PATH_INFO: `/v3/projects/bb9e349fd77e4d9aa3844bdf6f1108c8/users/e9464ccb65f3421489c604118a9e4d45/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:42.593312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6781675-cbf7-46d5-977a-2529ac57f7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:42.594175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6781675-cbf7-46d5-977a-2529ac57f7be admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:42.596130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6781675-cbf7-46d5-977a-2529ac57f7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:42.597974 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6781675-cbf7-46d5-977a-2529ac57f7be admin admin] PATH_INFO: `/v3/projects/9652c5e369af406691c19d84f92b9011/users/70c06ed9b31e43b48f6703397d67d100/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:42.602037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6781675-cbf7-46d5-977a-2529ac57f7be admin admin] RBAC: Authorizing `identity:create_grant(project_id=9652c5e369af406691c19d84f92b9011, user_id=70c06ed9b31e43b48f6703397d67d100, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:42.608138 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6781675-cbf7-46d5-977a-2529ac57f7be admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:42.617520 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:42.627572 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55268d-198b-4d10-8942-9afa56c343f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb9e349fd77e4d9aa3844bdf6f1108c8, user_id=e9464ccb65f3421489c604118a9e4d45, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:42.630299 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd55268d-198b-4d10-8942-9afa56c343f8 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:42.640075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1033/2070] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:42 2022] PUT /identity/v3/projects/9652c5e369af406691c19d84f92b9011/users/70c06ed9b31e43b48f6703397d67d100/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:42.650861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f032ce-5a23-44c0-b46f-08c23893384e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:42.652469 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f032ce-5a23-44c0-b46f-08c23893384e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:42.652754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f032ce-5a23-44c0-b46f-08c23893384e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:42.663934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1038/2071] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:42 2022] PUT /identity/v3/projects/bb9e349fd77e4d9aa3844bdf6f1108c8/users/e9464ccb65f3421489c604118a9e4d45/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:42.666578 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55268d-198b-4d10-8942-9afa56c343f8 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:42.691783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-18f032ce-5a23-44c0-b46f-08c23893384e None None] MFA Rules not processed for user `70c06ed9b31e43b48f6703397d67d100`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:42.692982 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-18f032ce-5a23-44c0-b46f-08c23893384e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:42.701407 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:42.814124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd55268d-198b-4d10-8942-9afa56c343f8 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:42.826569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930dc542-e0d8-4ede-9599-f6b2f7d59069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:42.830501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930dc542-e0d8-4ede-9599-f6b2f7d59069 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:42.831070 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930dc542-e0d8-4ede-9599-f6b2f7d59069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:42.831070 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930dc542-e0d8-4ede-9599-f6b2f7d59069 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:42.832529 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930dc542-e0d8-4ede-9599-f6b2f7d59069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:42.837339 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930dc542-e0d8-4ede-9599-f6b2f7d59069 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:42.857483 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1034/2072] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:42 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:42.869308 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f032ce-5a23-44c0-b46f-08c23893384e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:42.871494 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-18f032ce-5a23-44c0-b46f-08c23893384e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:42.882906 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1039/2073] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:13:42.886331 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930dc542-e0d8-4ede-9599-f6b2f7d59069 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:42.921528 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930dc542-e0d8-4ede-9599-f6b2f7d59069 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:42.960735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46081d42-f2a3-4361-b2cb-0aab1d365e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:42.962324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46081d42-f2a3-4361-b2cb-0aab1d365e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:42.962598 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46081d42-f2a3-4361-b2cb-0aab1d365e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:42.962866 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46081d42-f2a3-4361-b2cb-0aab1d365e0e admin admin] PATH_INFO: `/v3/projects/bb9e349fd77e4d9aa3844bdf6f1108c8/users/e9464ccb65f3421489c604118a9e4d45/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:42.968971 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46081d42-f2a3-4361-b2cb-0aab1d365e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb9e349fd77e4d9aa3844bdf6f1108c8, user_id=e9464ccb65f3421489c604118a9e4d45, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:42.971832 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46081d42-f2a3-4361-b2cb-0aab1d365e0e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:42.984724 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1040/2074] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:42 2022] PUT /identity/v3/projects/bb9e349fd77e4d9aa3844bdf6f1108c8/users/e9464ccb65f3421489c604118a9e4d45/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:42.986166 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f032ce-5a23-44c0-b46f-08c23893384e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:42.996132 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46081d42-f2a3-4361-b2cb-0aab1d365e0e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:42.998742 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-46081d42-f2a3-4361-b2cb-0aab1d365e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:43.044219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a0172-923b-4a2c-b463-97dcfe8014be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:43.044986 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a0172-923b-4a2c-b463-97dcfe8014be service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:43.045400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a0172-923b-4a2c-b463-97dcfe8014be service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:43.045989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a0172-923b-4a2c-b463-97dcfe8014be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:43.104241 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46081d42-f2a3-4361-b2cb-0aab1d365e0e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:43.115572 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642cffd-b40d-424b-841a-e8cad3484c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:43.116266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642cffd-b40d-424b-841a-e8cad3484c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:43.116479 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642cffd-b40d-424b-841a-e8cad3484c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:43.116746 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1642cffd-b40d-424b-841a-e8cad3484c38 admin admin] PATH_INFO: `/v3/projects/bab1237adca246ef9b82148b93c30e9b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:43.118213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642cffd-b40d-424b-841a-e8cad3484c38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bab1237adca246ef9b82148b93c30e9b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:43.120289 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1642cffd-b40d-424b-841a-e8cad3484c38 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:43.141564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a0172-923b-4a2c-b463-97dcfe8014be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:43.143902 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a0172-923b-4a2c-b463-97dcfe8014be service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:43.155894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1035/2075] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:42 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:13:43.157330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eab53c-9e17-4f4a-a7a8-bef2be0d937a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:43.157612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eab53c-9e17-4f4a-a7a8-bef2be0d937a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:43.157880 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eab53c-9e17-4f4a-a7a8-bef2be0d937a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:43.181053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:43.202999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-62eab53c-9e17-4f4a-a7a8-bef2be0d937a None None] MFA Rules not processed for user `e9464ccb65f3421489c604118a9e4d45`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:43.204242 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-62eab53c-9e17-4f4a-a7a8-bef2be0d937a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:43.206613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1642cffd-b40d-424b-841a-e8cad3484c38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bab1237adca246ef9b82148b93c30e9b', 'request_id': 'req-1642cffd-b40d-424b-841a-e8cad3484c38'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:43.240387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1642cffd-b40d-424b-841a-e8cad3484c38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bab1237adca246ef9b82148b93c30e9b', 'request_id': 'req-1642cffd-b40d-424b-841a-e8cad3484c38'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:43.257256 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1642cffd-b40d-424b-841a-e8cad3484c38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bab1237adca246ef9b82148b93c30e9b', 'request_id': 'req-1642cffd-b40d-424b-841a-e8cad3484c38'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:43.286759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1642cffd-b40d-424b-841a-e8cad3484c38 admin admin] The token cache is being invalidate because project bab1237adca246ef9b82148b93c30e9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:43.287089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1642cffd-b40d-424b-841a-e8cad3484c38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:43.287714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1642cffd-b40d-424b-841a-e8cad3484c38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:43.293478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1041/2076] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:42 2022] DELETE /identity/v3/projects/bab1237adca246ef9b82148b93c30e9b => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:43.294857 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642cffd-b40d-424b-841a-e8cad3484c38 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:43.295910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1642cffd-b40d-424b-841a-e8cad3484c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:43.301431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:43.312721 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1036/2077] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:43.315946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947121b2-7565-4a4f-b553-fdda29440d11 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:43.316497 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947121b2-7565-4a4f-b553-fdda29440d11 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:43.317003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947121b2-7565-4a4f-b553-fdda29440d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:43.341003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-947121b2-7565-4a4f-b553-fdda29440d11 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:43.342443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-947121b2-7565-4a4f-b553-fdda29440d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:43.382510 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1642cffd-b40d-424b-841a-e8cad3484c38 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:43.394928 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0c652-1929-4d42-8367-eda68fe8e9b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:43.395664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0c652-1929-4d42-8367-eda68fe8e9b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:43.395995 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0c652-1929-4d42-8367-eda68fe8e9b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:43.396393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0c652-1929-4d42-8367-eda68fe8e9b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:43.400026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8ff0c652-1929-4d42-8367-eda68fe8e9b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:43.433382 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1037/2078] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:43.440458 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aeb1c6-7161-4680-b243-2c63e904462f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:43.441917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aeb1c6-7161-4680-b243-2c63e904462f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:43.442450 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aeb1c6-7161-4680-b243-2c63e904462f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:43.468712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c5aeb1c6-7161-4680-b243-2c63e904462f None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:43.470112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c5aeb1c6-7161-4680-b243-2c63e904462f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:43.486701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1038/2079] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:43.491834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aeb1c6-7161-4680-b243-2c63e904462f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:43.494371 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0c652-1929-4d42-8367-eda68fe8e9b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:43.495841 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff0c652-1929-4d42-8367-eda68fe8e9b9 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:43.511620 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aeb1c6-7161-4680-b243-2c63e904462f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:43.516662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1042/2080] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:43.519109 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd4e2b-928f-4479-a3ad-e85053aa2271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:43.520127 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd4e2b-928f-4479-a3ad-e85053aa2271 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:43.520550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd4e2b-928f-4479-a3ad-e85053aa2271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:43.520964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cd4e2b-928f-4479-a3ad-e85053aa2271 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:43.522094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd4e2b-928f-4479-a3ad-e85053aa2271 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:13:43.522536 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd4e2b-928f-4479-a3ad-e85053aa2271 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:43.536070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cd4e2b-928f-4479-a3ad-e85053aa2271 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:43.554084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0c652-1929-4d42-8367-eda68fe8e9b9 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:43.556247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8ff0c652-1929-4d42-8367-eda68fe8e9b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:43.579282 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0c652-1929-4d42-8367-eda68fe8e9b9 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:43.579530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1039/2081] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:13:43.584855 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd4e2b-928f-4479-a3ad-e85053aa2271 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:43.589938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b77c6-fc6d-4541-8379-a474a378c56f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:43.590694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b77c6-fc6d-4541-8379-a474a378c56f service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:43.591355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b77c6-fc6d-4541-8379-a474a378c56f service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:43.592812 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b77c6-fc6d-4541-8379-a474a378c56f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:43.595802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e61b77c6-fc6d-4541-8379-a474a378c56f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:43.604723 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cd4e2b-928f-4479-a3ad-e85053aa2271 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:43.614484 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8bbe1-5e38-4ce5-b76f-2d10dab469e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:43.615319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8bbe1-5e38-4ce5-b76f-2d10dab469e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:43.615609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8bbe1-5e38-4ce5-b76f-2d10dab469e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:43.616009 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8bbe1-5e38-4ce5-b76f-2d10dab469e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:43.619737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8bbe1-5e38-4ce5-b76f-2d10dab469e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:43.627263 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8bbe1-5e38-4ce5-b76f-2d10dab469e8 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:43.729464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1040/2082] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:43 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:13:43.734513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8bbe1-5e38-4ce5-b76f-2d10dab469e8 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:43.766254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b77c6-fc6d-4541-8379-a474a378c56f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:43.769376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b77c6-fc6d-4541-8379-a474a378c56f service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:43.790386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1043/2083] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:13:43.891489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:43.909762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8bbe1-5e38-4ce5-b76f-2d10dab469e8 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:43.924512 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406ed4d-1477-46fa-9803-a25aac83ffc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:43.927303 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406ed4d-1477-46fa-9803-a25aac83ffc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:43.927303 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406ed4d-1477-46fa-9803-a25aac83ffc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:43.927483 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4406ed4d-1477-46fa-9803-a25aac83ffc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:43.927990 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406ed4d-1477-46fa-9803-a25aac83ffc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:43.929978 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4406ed4d-1477-46fa-9803-a25aac83ffc9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:43.956207 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1041/2084] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:43 2022] POST /identity/v3/users => generated 291 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:43.960707 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b77c6-fc6d-4541-8379-a474a378c56f service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:43.981037 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b77c6-fc6d-4541-8379-a474a378c56f service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:43.991678 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c268bee-0b44-45e7-b09c-2280937821d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:43.993001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c268bee-0b44-45e7-b09c-2280937821d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:43.993580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c268bee-0b44-45e7-b09c-2280937821d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:43.994856 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c268bee-0b44-45e7-b09c-2280937821d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:43.996026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c268bee-0b44-45e7-b09c-2280937821d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:44.000670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c268bee-0b44-45e7-b09c-2280937821d7 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:44.024845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1044/2085] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:44.030108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406ed4d-1477-46fa-9803-a25aac83ffc9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:44.049033 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4406ed4d-1477-46fa-9803-a25aac83ffc9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:44.057147 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce11bdf-aa9f-42bd-9f9e-bc7773746b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:44.058122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce11bdf-aa9f-42bd-9f9e-bc7773746b32 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:44.058554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce11bdf-aa9f-42bd-9f9e-bc7773746b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:44.058900 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce11bdf-aa9f-42bd-9f9e-bc7773746b32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:44.060476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce11bdf-aa9f-42bd-9f9e-bc7773746b32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:44.062843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce11bdf-aa9f-42bd-9f9e-bc7773746b32 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:44.095573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1042/2086] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:44.100175 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c268bee-0b44-45e7-b09c-2280937821d7 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:44.113297 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c268bee-0b44-45e7-b09c-2280937821d7 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:44.120032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37c6f6-8eec-490a-9b80-6d9e29cd0966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:44.120812 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37c6f6-8eec-490a-9b80-6d9e29cd0966 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:44.122410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37c6f6-8eec-490a-9b80-6d9e29cd0966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:44.122705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37c6f6-8eec-490a-9b80-6d9e29cd0966 admin admin] PATH_INFO: `/v3/projects/b18fac49577a4056bf6c5bbfed781f11/users/ed922e856cad4be19467bc2a998d576f/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:44.144799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37c6f6-8eec-490a-9b80-6d9e29cd0966 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b18fac49577a4056bf6c5bbfed781f11, user_id=ed922e856cad4be19467bc2a998d576f, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:44.147459 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37c6f6-8eec-490a-9b80-6d9e29cd0966 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:44.162253 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce11bdf-aa9f-42bd-9f9e-bc7773746b32 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:44.164145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6ce11bdf-aa9f-42bd-9f9e-bc7773746b32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:44.173003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1045/2087] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:44 2022] PUT /identity/v3/projects/b18fac49577a4056bf6c5bbfed781f11/users/ed922e856cad4be19467bc2a998d576f/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:44.197697 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2cefd-6b00-455d-a060-049e6cb92635 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:44.198839 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2cefd-6b00-455d-a060-049e6cb92635 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:44.199822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2cefd-6b00-455d-a060-049e6cb92635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:44.219019 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce11bdf-aa9f-42bd-9f9e-bc7773746b32 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:44.269709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-bcc2cefd-6b00-455d-a060-049e6cb92635 None None] MFA Rules not processed for user `ed922e856cad4be19467bc2a998d576f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:44.270999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bcc2cefd-6b00-455d-a060-049e6cb92635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:44.287486 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:44.356910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23fbf9-8f12-42bf-a7d1-2eeffea8f6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:44.357891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23fbf9-8f12-42bf-a7d1-2eeffea8f6e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:44.358182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23fbf9-8f12-42bf-a7d1-2eeffea8f6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:44.358505 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23fbf9-8f12-42bf-a7d1-2eeffea8f6e1 admin admin] PATH_INFO: `/v3/users/9423f778e1db4fd8af9a3ff5ad0f49c0` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:44.359816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23fbf9-8f12-42bf-a7d1-2eeffea8f6e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9423f778e1db4fd8af9a3ff5ad0f49c0)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:44.362509 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf23fbf9-8f12-42bf-a7d1-2eeffea8f6e1 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:44.424977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1046/2088] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:44.426098 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2cefd-6b00-455d-a060-049e6cb92635 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:44.452239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2cefd-6b00-455d-a060-049e6cb92635 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:44.465162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be792b-091b-4bc3-955c-2de9017d2e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:44.465930 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be792b-091b-4bc3-955c-2de9017d2e59 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:44.466285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be792b-091b-4bc3-955c-2de9017d2e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:44.466706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8be792b-091b-4bc3-955c-2de9017d2e59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:44.467419 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be792b-091b-4bc3-955c-2de9017d2e59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:44.473410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8be792b-091b-4bc3-955c-2de9017d2e59 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:44.481676 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bf23fbf9-8f12-42bf-a7d1-2eeffea8f6e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9423f778e1db4fd8af9a3ff5ad0f49c0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:44.498704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bf23fbf9-8f12-42bf-a7d1-2eeffea8f6e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9423f778e1db4fd8af9a3ff5ad0f49c0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:44.528935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bf23fbf9-8f12-42bf-a7d1-2eeffea8f6e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9423f778e1db4fd8af9a3ff5ad0f49c0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:44.530219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bf23fbf9-8f12-42bf-a7d1-2eeffea8f6e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9423f778e1db4fd8af9a3ff5ad0f49c0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:44.532288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bf23fbf9-8f12-42bf-a7d1-2eeffea8f6e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9423f778e1db4fd8af9a3ff5ad0f49c0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:44.542005 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1047/2089] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:44 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:44.546993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be792b-091b-4bc3-955c-2de9017d2e59 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:44.548884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f8be792b-091b-4bc3-955c-2de9017d2e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:44.564708 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1043/2090] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:44 2022] DELETE /identity/v3/users/9423f778e1db4fd8af9a3ff5ad0f49c0 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:13:44.566663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e9b7f-2aa1-4928-912c-2bb74c430781 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:44.566920 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e9b7f-2aa1-4928-912c-2bb74c430781 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:44.567170 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e9b7f-2aa1-4928-912c-2bb74c430781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:44.610248 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c38e9b7f-2aa1-4928-912c-2bb74c430781 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:44.611724 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c38e9b7f-2aa1-4928-912c-2bb74c430781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:44.670481 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8be792b-091b-4bc3-955c-2de9017d2e59 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:44.713073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:44.742033 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1044/2091] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:44.795618 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e9b7f-2aa1-4928-912c-2bb74c430781 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:44.799747 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c38e9b7f-2aa1-4928-912c-2bb74c430781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:44.838162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d43b20-4aee-4944-9251-e1414da65ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:44.838792 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d43b20-4aee-4944-9251-e1414da65ed4 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:44.839073 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d43b20-4aee-4944-9251-e1414da65ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:44.839334 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d43b20-4aee-4944-9251-e1414da65ed4 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:44.843097 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d43b20-4aee-4944-9251-e1414da65ed4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:44.851008 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d43b20-4aee-4944-9251-e1414da65ed4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:44.904151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1048/2092] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:44 2022] POST /identity/v3/users => generated 297 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:44.908467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d43b20-4aee-4944-9251-e1414da65ed4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:44.917122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e9b7f-2aa1-4928-912c-2bb74c430781 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:44.923762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85299e-a1c1-4d1f-bd5f-49cf2ec6fc6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:44.924047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85299e-a1c1-4d1f-bd5f-49cf2ec6fc6a service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:44.924305 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85299e-a1c1-4d1f-bd5f-49cf2ec6fc6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:44.924576 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb85299e-a1c1-4d1f-bd5f-49cf2ec6fc6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:44.932344 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d43b20-4aee-4944-9251-e1414da65ed4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:44.935986 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85299e-a1c1-4d1f-bd5f-49cf2ec6fc6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:44.937697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb85299e-a1c1-4d1f-bd5f-49cf2ec6fc6a service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:44.944418 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c7806-b97f-42e2-b370-2953151c7dda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:44.946760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c7806-b97f-42e2-b370-2953151c7dda admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:44.947147 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c7806-b97f-42e2-b370-2953151c7dda admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:44.947390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64c7806-b97f-42e2-b370-2953151c7dda admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:44.949114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c7806-b97f-42e2-b370-2953151c7dda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:44.952544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1045/2093] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:13:44.954420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64c7806-b97f-42e2-b370-2953151c7dda admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:44.985590 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1049/2094] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:44.987774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85299e-a1c1-4d1f-bd5f-49cf2ec6fc6a service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:45.001690 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb85299e-a1c1-4d1f-bd5f-49cf2ec6fc6a service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:45.007295 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c168701-93eb-489f-a916-3c4b6a6f74e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:45.008175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c168701-93eb-489f-a916-3c4b6a6f74e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:45.009052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c168701-93eb-489f-a916-3c4b6a6f74e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:45.009439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c168701-93eb-489f-a916-3c4b6a6f74e3 admin admin] PATH_INFO: `/v3/projects/c8bb0277129c475e96b8daecd58a3386/users/4abcec2a15fa42fe8ec5228523afa4e4/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:45.031722 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c168701-93eb-489f-a916-3c4b6a6f74e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8bb0277129c475e96b8daecd58a3386, user_id=4abcec2a15fa42fe8ec5228523afa4e4, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:45.034986 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c168701-93eb-489f-a916-3c4b6a6f74e3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:45.061376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1046/2095] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:44 2022] PUT /identity/v3/projects/c8bb0277129c475e96b8daecd58a3386/users/4abcec2a15fa42fe8ec5228523afa4e4/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:45.065460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c7806-b97f-42e2-b370-2953151c7dda admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:45.115620 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64c7806-b97f-42e2-b370-2953151c7dda admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:45.141037 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:45.273963 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9035c-7e98-42a8-98b3-4d7b47e45e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:45.275281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9035c-7e98-42a8-98b3-4d7b47e45e49 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:45.275559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9035c-7e98-42a8-98b3-4d7b47e45e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:45.275876 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9035c-7e98-42a8-98b3-4d7b47e45e49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:45.288405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9035c-7e98-42a8-98b3-4d7b47e45e49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:45.294805 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9035c-7e98-42a8-98b3-4d7b47e45e49 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:45.315827 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1050/2096] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:45 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:13:45.319551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c168701-93eb-489f-a916-3c4b6a6f74e3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:45.337762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c168701-93eb-489f-a916-3c4b6a6f74e3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:45.346108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a9938-fa43-450c-8821-7dd92f9414fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:45.347199 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a9938-fa43-450c-8821-7dd92f9414fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:45.347626 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a9938-fa43-450c-8821-7dd92f9414fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:45.348051 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55a9938-fa43-450c-8821-7dd92f9414fa admin admin] PATH_INFO: `/v3/projects/c8bb0277129c475e96b8daecd58a3386/users/4abcec2a15fa42fe8ec5228523afa4e4/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:45.350749 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a9938-fa43-450c-8821-7dd92f9414fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8bb0277129c475e96b8daecd58a3386, user_id=4abcec2a15fa42fe8ec5228523afa4e4, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:45.368945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55a9938-fa43-450c-8821-7dd92f9414fa admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:45.388513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1047/2097] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:45 2022] PUT /identity/v3/projects/c8bb0277129c475e96b8daecd58a3386/users/4abcec2a15fa42fe8ec5228523afa4e4/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:45.399946 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298ebe7-06b0-420d-8d73-b97fecf181f4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:45.400277 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298ebe7-06b0-420d-8d73-b97fecf181f4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:45.400751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298ebe7-06b0-420d-8d73-b97fecf181f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:45.442621 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e298ebe7-06b0-420d-8d73-b97fecf181f4 None None] MFA Rules not processed for user `4abcec2a15fa42fe8ec5228523afa4e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:45.443846 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e298ebe7-06b0-420d-8d73-b97fecf181f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:45.556050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1051/2098] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:45.619854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a9938-fa43-450c-8821-7dd92f9414fa admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:45.624809 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e55a9938-fa43-450c-8821-7dd92f9414fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:45.731718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298ebe7-06b0-420d-8d73-b97fecf181f4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:45.734489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e298ebe7-06b0-420d-8d73-b97fecf181f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:45.756611 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55a9938-fa43-450c-8821-7dd92f9414fa admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:45.773220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaaea4-8c03-4fbf-89c1-34ddaf8d535c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:45.773865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaaea4-8c03-4fbf-89c1-34ddaf8d535c service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:45.774375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaaea4-8c03-4fbf-89c1-34ddaf8d535c service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:45.774642 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadaaea4-8c03-4fbf-89c1-34ddaf8d535c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:45.785832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadaaea4-8c03-4fbf-89c1-34ddaf8d535c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:45.787321 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadaaea4-8c03-4fbf-89c1-34ddaf8d535c service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:45.799886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1048/2099] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:45 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:45.870931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298ebe7-06b0-420d-8d73-b97fecf181f4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:45.877878 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5259a46-b318-4a59-864d-889c3803b628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:45.878182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5259a46-b318-4a59-864d-889c3803b628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:45.878686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5259a46-b318-4a59-864d-889c3803b628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:45.878907 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5259a46-b318-4a59-864d-889c3803b628 admin admin] PATH_INFO: `/v3/projects/05a68c956c924e8191b44aa24377a725` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:45.880276 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5259a46-b318-4a59-864d-889c3803b628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05a68c956c924e8191b44aa24377a725)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:45.882380 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5259a46-b318-4a59-864d-889c3803b628 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:45.906940 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaaea4-8c03-4fbf-89c1-34ddaf8d535c service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:45.918909 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadaaea4-8c03-4fbf-89c1-34ddaf8d535c service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:45.928137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63c448-39f5-46f9-a561-b8dfcb3eedb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:45.929213 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63c448-39f5-46f9-a561-b8dfcb3eedb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:45.929500 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63c448-39f5-46f9-a561-b8dfcb3eedb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:45.929767 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce63c448-39f5-46f9-a561-b8dfcb3eedb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:45.932074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ce63c448-39f5-46f9-a561-b8dfcb3eedb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:45.938321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:45.950986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5259a46-b318-4a59-864d-889c3803b628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05a68c956c924e8191b44aa24377a725', 'request_id': 'req-f5259a46-b318-4a59-864d-889c3803b628'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:45.959088 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5259a46-b318-4a59-864d-889c3803b628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05a68c956c924e8191b44aa24377a725', 'request_id': 'req-f5259a46-b318-4a59-864d-889c3803b628'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:45.965224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5259a46-b318-4a59-864d-889c3803b628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05a68c956c924e8191b44aa24377a725', 'request_id': 'req-f5259a46-b318-4a59-864d-889c3803b628'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:45.979188 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5259a46-b318-4a59-864d-889c3803b628 admin admin] The token cache is being invalidate because project 05a68c956c924e8191b44aa24377a725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:45.979484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5259a46-b318-4a59-864d-889c3803b628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:45.980153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5259a46-b318-4a59-864d-889c3803b628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:45.981709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1052/2100] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:45 2022] DELETE /identity/v3/projects/05a68c956c924e8191b44aa24377a725 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:45.985726 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5259a46-b318-4a59-864d-889c3803b628 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:45.987075 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f5259a46-b318-4a59-864d-889c3803b628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:46.002240 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63c448-39f5-46f9-a561-b8dfcb3eedb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:46.003641 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce63c448-39f5-46f9-a561-b8dfcb3eedb5 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:46.007565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:46.015275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1049/2101] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:45 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:13:46.072892 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5259a46-b318-4a59-864d-889c3803b628 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:46.078701 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5497b2b-a34b-4165-9e11-3cede156f015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:46.079348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5497b2b-a34b-4165-9e11-3cede156f015 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:46.079641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5497b2b-a34b-4165-9e11-3cede156f015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:46.079921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5497b2b-a34b-4165-9e11-3cede156f015 admin admin] PATH_INFO: `/v3/users/70c06ed9b31e43b48f6703397d67d100` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:46.081154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5497b2b-a34b-4165-9e11-3cede156f015 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70c06ed9b31e43b48f6703397d67d100)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:46.083243 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5497b2b-a34b-4165-9e11-3cede156f015 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:46.143927 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c5497b2b-a34b-4165-9e11-3cede156f015 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70c06ed9b31e43b48f6703397d67d100'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:46.159244 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c5497b2b-a34b-4165-9e11-3cede156f015 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70c06ed9b31e43b48f6703397d67d100'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:46.182671 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c5497b2b-a34b-4165-9e11-3cede156f015 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70c06ed9b31e43b48f6703397d67d100'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:46.183334 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c5497b2b-a34b-4165-9e11-3cede156f015 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70c06ed9b31e43b48f6703397d67d100'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:46.196288 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c5497b2b-a34b-4165-9e11-3cede156f015 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70c06ed9b31e43b48f6703397d67d100'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:46.202450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1053/2102] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:45 2022] DELETE /identity/v3/users/70c06ed9b31e43b48f6703397d67d100 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:13:46.486386 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63c448-39f5-46f9-a561-b8dfcb3eedb5 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:46.498052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce63c448-39f5-46f9-a561-b8dfcb3eedb5 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:46.557275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede21511-1f56-4776-bb4d-34314719b203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:46.557897 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede21511-1f56-4776-bb4d-34314719b203 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:46.558164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede21511-1f56-4776-bb4d-34314719b203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:46.558429 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede21511-1f56-4776-bb4d-34314719b203 admin admin] PATH_INFO: `/v3/projects/9652c5e369af406691c19d84f92b9011` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:46.559513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede21511-1f56-4776-bb4d-34314719b203 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9652c5e369af406691c19d84f92b9011)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:46.561424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede21511-1f56-4776-bb4d-34314719b203 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:46.606800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ede21511-1f56-4776-bb4d-34314719b203 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9652c5e369af406691c19d84f92b9011', 'request_id': 'req-ede21511-1f56-4776-bb4d-34314719b203'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:46.613612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ede21511-1f56-4776-bb4d-34314719b203 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9652c5e369af406691c19d84f92b9011', 'request_id': 'req-ede21511-1f56-4776-bb4d-34314719b203'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:46.620359 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ede21511-1f56-4776-bb4d-34314719b203 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9652c5e369af406691c19d84f92b9011', 'request_id': 'req-ede21511-1f56-4776-bb4d-34314719b203'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:46.631595 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ede21511-1f56-4776-bb4d-34314719b203 admin admin] The token cache is being invalidate because project 9652c5e369af406691c19d84f92b9011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:46.631883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ede21511-1f56-4776-bb4d-34314719b203 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:46.632516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ede21511-1f56-4776-bb4d-34314719b203 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:46.634567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1050/2103] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:46 2022] DELETE /identity/v3/projects/9652c5e369af406691c19d84f92b9011 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:46.639294 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a3d49-5cec-42c0-8644-d52e0c47ec40 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:46.639578 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a3d49-5cec-42c0-8644-d52e0c47ec40 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:46.639853 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a3d49-5cec-42c0-8644-d52e0c47ec40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:46.659486 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-666a3d49-5cec-42c0-8644-d52e0c47ec40 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:46.660679 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-666a3d49-5cec-42c0-8644-d52e0c47ec40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:46.700339 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:46.720996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1054/2104] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:46.729670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2e982-4694-4a58-b5bc-4cb64a130da9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:46.729957 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2e982-4694-4a58-b5bc-4cb64a130da9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:46.730234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb2e982-4694-4a58-b5bc-4cb64a130da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:46.749959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-1eb2e982-4694-4a58-b5bc-4cb64a130da9 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:46.751158 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1eb2e982-4694-4a58-b5bc-4cb64a130da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:46.759486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1051/2105] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:46.763604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a3d49-5cec-42c0-8644-d52e0c47ec40 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:46.773163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a3d49-5cec-42c0-8644-d52e0c47ec40 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:46.778286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f4c8c-fe73-4fc7-bb3d-ce981c42874a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:46.778870 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f4c8c-fe73-4fc7-bb3d-ce981c42874a admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:46.779146 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f4c8c-fe73-4fc7-bb3d-ce981c42874a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:46.779415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58f4c8c-fe73-4fc7-bb3d-ce981c42874a admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:46.780077 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f4c8c-fe73-4fc7-bb3d-ce981c42874a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:13:46.780372 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f4c8c-fe73-4fc7-bb3d-ce981c42874a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:46.781979 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58f4c8c-fe73-4fc7-bb3d-ce981c42874a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:46.793228 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1055/2106] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:13:46.797400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2e982-4694-4a58-b5bc-4cb64a130da9 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:46.807190 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb2e982-4694-4a58-b5bc-4cb64a130da9 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:46.811877 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fa927-1956-4cb4-9585-080486a16b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:46.812471 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fa927-1956-4cb4-9585-080486a16b7e admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:46.812751 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fa927-1956-4cb4-9585-080486a16b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:46.813022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87fa927-1956-4cb4-9585-080486a16b7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:46.813732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fa927-1956-4cb4-9585-080486a16b7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:46.815581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87fa927-1956-4cb4-9585-080486a16b7e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:46.843515 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1052/2107] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:46 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:46.850310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f4c8c-fe73-4fc7-bb3d-ce981c42874a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:46.861310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58f4c8c-fe73-4fc7-bb3d-ce981c42874a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:46.920365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09095c-2526-4004-baad-47171663b3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:46.920967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09095c-2526-4004-baad-47171663b3bb admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:46.921267 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09095c-2526-4004-baad-47171663b3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:46.921537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b09095c-2526-4004-baad-47171663b3bb admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:46.922151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09095c-2526-4004-baad-47171663b3bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:46.924024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b09095c-2526-4004-baad-47171663b3bb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:46.945664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1056/2108] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:46 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:46.950242 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fa927-1956-4cb4-9585-080486a16b7e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:46.961469 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87fa927-1956-4cb4-9585-080486a16b7e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:46.967660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b71854-2b86-4ac1-9263-07d9c979fb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:46.968290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b71854-2b86-4ac1-9263-07d9c979fb65 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:46.968567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b71854-2b86-4ac1-9263-07d9c979fb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:46.968838 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b71854-2b86-4ac1-9263-07d9c979fb65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:46.969559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b71854-2b86-4ac1-9263-07d9c979fb65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:46.971224 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b71854-2b86-4ac1-9263-07d9c979fb65 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:46.983486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1053/2109] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:46.987497 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09095c-2526-4004-baad-47171663b3bb admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:47.000243 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b09095c-2526-4004-baad-47171663b3bb admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:47.005043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d025276-34bd-4e7c-8eb8-bd743a3769b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:47.005681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d025276-34bd-4e7c-8eb8-bd743a3769b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:47.005954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d025276-34bd-4e7c-8eb8-bd743a3769b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:47.006271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d025276-34bd-4e7c-8eb8-bd743a3769b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:47.006909 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d025276-34bd-4e7c-8eb8-bd743a3769b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:47.009101 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d025276-34bd-4e7c-8eb8-bd743a3769b3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:47.020740 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1057/2110] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:47.024940 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b71854-2b86-4ac1-9263-07d9c979fb65 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:47.035906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b71854-2b86-4ac1-9263-07d9c979fb65 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:47.041271 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1cde6a-e2fa-4b9c-8e45-ba171853b1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:47.042077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cde6a-e2fa-4b9c-8e45-ba171853b1e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:47.042421 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cde6a-e2fa-4b9c-8e45-ba171853b1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:47.042730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1cde6a-e2fa-4b9c-8e45-ba171853b1e0 admin admin] PATH_INFO: `/v3/projects/0fa35464b444471893c8d6a60ac2376e/users/847ef2b4a425409eb2fb9d3c9937e910/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:47.060576 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1cde6a-e2fa-4b9c-8e45-ba171853b1e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fa35464b444471893c8d6a60ac2376e, user_id=847ef2b4a425409eb2fb9d3c9937e910, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:47.063620 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1cde6a-e2fa-4b9c-8e45-ba171853b1e0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:47.080236 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1054/2111] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:47 2022] PUT /identity/v3/projects/0fa35464b444471893c8d6a60ac2376e/users/847ef2b4a425409eb2fb9d3c9937e910/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:13:47.091047 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8555b-7415-4ad4-8dd0-8f742cdba547 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:47.091550 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8555b-7415-4ad4-8dd0-8f742cdba547 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:47.091899 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b8555b-7415-4ad4-8dd0-8f742cdba547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:47.121806 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f5b8555b-7415-4ad4-8dd0-8f742cdba547 None None] MFA Rules not processed for user `847ef2b4a425409eb2fb9d3c9937e910`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:47.122949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f5b8555b-7415-4ad4-8dd0-8f742cdba547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:47.190647 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1058/2112] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:47.203660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1cde6a-e2fa-4b9c-8e45-ba171853b1e0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:47.205460 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1a1cde6a-e2fa-4b9c-8e45-ba171853b1e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:47.237752 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:47.273163 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1cde6a-e2fa-4b9c-8e45-ba171853b1e0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:47.277160 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a0be9-4f87-48fc-8d4b-4d29aad5aeeb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:47.277872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a0be9-4f87-48fc-8d4b-4d29aad5aeeb service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:47.278224 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a0be9-4f87-48fc-8d4b-4d29aad5aeeb service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:47.278570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a0be9-4f87-48fc-8d4b-4d29aad5aeeb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:47.287750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a0be9-4f87-48fc-8d4b-4d29aad5aeeb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:47.289377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a0be9-4f87-48fc-8d4b-4d29aad5aeeb service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:47.300801 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1055/2113] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:47.330067 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8555b-7415-4ad4-8dd0-8f742cdba547 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:47.385495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:47.407517 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b8555b-7415-4ad4-8dd0-8f742cdba547 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:47.412953 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e43d52-3a42-4e80-9aee-1a3dedf5d0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:47.413633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e43d52-3a42-4e80-9aee-1a3dedf5d0a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:47.413911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e43d52-3a42-4e80-9aee-1a3dedf5d0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:47.414177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e43d52-3a42-4e80-9aee-1a3dedf5d0a1 admin admin] PATH_INFO: `/v3/users/847ef2b4a425409eb2fb9d3c9937e910` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:47.416932 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e43d52-3a42-4e80-9aee-1a3dedf5d0a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=847ef2b4a425409eb2fb9d3c9937e910)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:47.418873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e43d52-3a42-4e80-9aee-1a3dedf5d0a1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:47.473173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f7e43d52-3a42-4e80-9aee-1a3dedf5d0a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '847ef2b4a425409eb2fb9d3c9937e910'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:47.481150 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f7e43d52-3a42-4e80-9aee-1a3dedf5d0a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '847ef2b4a425409eb2fb9d3c9937e910'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:47.489037 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f7e43d52-3a42-4e80-9aee-1a3dedf5d0a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '847ef2b4a425409eb2fb9d3c9937e910'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:47.489710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f7e43d52-3a42-4e80-9aee-1a3dedf5d0a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '847ef2b4a425409eb2fb9d3c9937e910'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:47.503377 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f7e43d52-3a42-4e80-9aee-1a3dedf5d0a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '847ef2b4a425409eb2fb9d3c9937e910'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:47.512312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1059/2114] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:47 2022] DELETE /identity/v3/users/847ef2b4a425409eb2fb9d3c9937e910 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:47.517349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616a5b1-4a80-4ae1-a1ee-25fbc94b7145 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:47.517644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616a5b1-4a80-4ae1-a1ee-25fbc94b7145 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:47.517925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9616a5b1-4a80-4ae1-a1ee-25fbc94b7145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:47.544541 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9616a5b1-4a80-4ae1-a1ee-25fbc94b7145 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:47.545779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9616a5b1-4a80-4ae1-a1ee-25fbc94b7145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:47.619471 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1056/2115] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:47.629319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e43d52-3a42-4e80-9aee-1a3dedf5d0a1 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:47.630765 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f7e43d52-3a42-4e80-9aee-1a3dedf5d0a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:47.688370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e43d52-3a42-4e80-9aee-1a3dedf5d0a1 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:47.691858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ff821-ba70-48eb-9b6a-60f1088e706b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:47.692447 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ff821-ba70-48eb-9b6a-60f1088e706b service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:47.692721 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ff821-ba70-48eb-9b6a-60f1088e706b service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:47.692986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ff821-ba70-48eb-9b6a-60f1088e706b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:47.701756 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ff821-ba70-48eb-9b6a-60f1088e706b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:47.703151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ff821-ba70-48eb-9b6a-60f1088e706b service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:47.713708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1060/2116] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:47.978735 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616a5b1-4a80-4ae1-a1ee-25fbc94b7145 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:47.980191 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9616a5b1-4a80-4ae1-a1ee-25fbc94b7145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:47.992354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9616a5b1-4a80-4ae1-a1ee-25fbc94b7145 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:47.997859 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea05072-8769-4ec1-86d8-5a9c45b9f4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:47.998590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea05072-8769-4ec1-86d8-5a9c45b9f4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:47.998976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea05072-8769-4ec1-86d8-5a9c45b9f4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:47.999256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea05072-8769-4ec1-86d8-5a9c45b9f4ba admin admin] PATH_INFO: `/v3/projects/0fa35464b444471893c8d6a60ac2376e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:48.000375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea05072-8769-4ec1-86d8-5a9c45b9f4ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fa35464b444471893c8d6a60ac2376e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:48.002544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea05072-8769-4ec1-86d8-5a9c45b9f4ba admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:48.050243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bea05072-8769-4ec1-86d8-5a9c45b9f4ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fa35464b444471893c8d6a60ac2376e', 'request_id': 'req-bea05072-8769-4ec1-86d8-5a9c45b9f4ba'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:48.059150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bea05072-8769-4ec1-86d8-5a9c45b9f4ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fa35464b444471893c8d6a60ac2376e', 'request_id': 'req-bea05072-8769-4ec1-86d8-5a9c45b9f4ba'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:48.067160 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bea05072-8769-4ec1-86d8-5a9c45b9f4ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fa35464b444471893c8d6a60ac2376e', 'request_id': 'req-bea05072-8769-4ec1-86d8-5a9c45b9f4ba'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:48.080275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bea05072-8769-4ec1-86d8-5a9c45b9f4ba admin admin] The token cache is being invalidate because project 0fa35464b444471893c8d6a60ac2376e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:48.080674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bea05072-8769-4ec1-86d8-5a9c45b9f4ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:48.081438 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bea05072-8769-4ec1-86d8-5a9c45b9f4ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:48.083067 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1057/2117] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:47 2022] DELETE /identity/v3/projects/0fa35464b444471893c8d6a60ac2376e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:48.088765 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d86a98-ca2f-411a-90be-997f239ab45e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:48.089250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d86a98-ca2f-411a-90be-997f239ab45e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:48.089344 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d86a98-ca2f-411a-90be-997f239ab45e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:48.108967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-36d86a98-ca2f-411a-90be-997f239ab45e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:48.110181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-36d86a98-ca2f-411a-90be-997f239ab45e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:48.176687 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1061/2118] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:13:48.184286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb673-132b-42db-83bf-aacb685e8d51 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:48.184568 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb673-132b-42db-83bf-aacb685e8d51 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:48.184838 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebffb673-132b-42db-83bf-aacb685e8d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:48.204915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ebffb673-132b-42db-83bf-aacb685e8d51 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:48.206149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ebffb673-132b-42db-83bf-aacb685e8d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:48.215140 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1058/2119] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:48.219435 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d86a98-ca2f-411a-90be-997f239ab45e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:48.224407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:48.231026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d86a98-ca2f-411a-90be-997f239ab45e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:48.236890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd49b9-a31f-4207-82e0-9278bb4fdcab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:48.237531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd49b9-a31f-4207-82e0-9278bb4fdcab admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:48.237810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd49b9-a31f-4207-82e0-9278bb4fdcab admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:48.238081 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd49b9-a31f-4207-82e0-9278bb4fdcab admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:48.238720 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd49b9-a31f-4207-82e0-9278bb4fdcab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:13:48.239021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd49b9-a31f-4207-82e0-9278bb4fdcab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:48.240633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd49b9-a31f-4207-82e0-9278bb4fdcab admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:48.252218 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1062/2120] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:13:48.256523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffb673-132b-42db-83bf-aacb685e8d51 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:48.266477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebffb673-132b-42db-83bf-aacb685e8d51 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:48.271360 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2bd60-1863-4b93-b7fd-a68e08e9c4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:48.271952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2bd60-1863-4b93-b7fd-a68e08e9c4ea admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:48.272224 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2bd60-1863-4b93-b7fd-a68e08e9c4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:48.272489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2bd60-1863-4b93-b7fd-a68e08e9c4ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:48.273113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2bd60-1863-4b93-b7fd-a68e08e9c4ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:48.275645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2bd60-1863-4b93-b7fd-a68e08e9c4ea admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:48.302664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1059/2121] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:48 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:48.307492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd49b9-a31f-4207-82e0-9278bb4fdcab admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:48.317487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd49b9-a31f-4207-82e0-9278bb4fdcab admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:48.374199 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cacc9b-31ab-4073-a265-3e50ed03bdc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:48.374801 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cacc9b-31ab-4073-a265-3e50ed03bdc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:48.375079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cacc9b-31ab-4073-a265-3e50ed03bdc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:48.375379 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cacc9b-31ab-4073-a265-3e50ed03bdc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:48.375998 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cacc9b-31ab-4073-a265-3e50ed03bdc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:48.377992 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cacc9b-31ab-4073-a265-3e50ed03bdc2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:48.406935 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1063/2122] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:48 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:13:48.411536 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2bd60-1863-4b93-b7fd-a68e08e9c4ea admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:48.422450 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2bd60-1863-4b93-b7fd-a68e08e9c4ea admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:48.427366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ea680-271a-426c-863b-5d960a863beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:48.428058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ea680-271a-426c-863b-5d960a863beb admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:48.428400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ea680-271a-426c-863b-5d960a863beb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:48.428736 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446ea680-271a-426c-863b-5d960a863beb admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:48.429480 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ea680-271a-426c-863b-5d960a863beb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:48.431610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446ea680-271a-426c-863b-5d960a863beb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:48.441840 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1060/2123] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:13:48.445757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cacc9b-31ab-4073-a265-3e50ed03bdc2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:48.456391 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cacc9b-31ab-4073-a265-3e50ed03bdc2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:48.461210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c492eb-b5f3-402c-b739-6f8a5bd22ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:48.461967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c492eb-b5f3-402c-b739-6f8a5bd22ba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:48.462222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c492eb-b5f3-402c-b739-6f8a5bd22ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:48.462638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c492eb-b5f3-402c-b739-6f8a5bd22ba6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:48.463282 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c492eb-b5f3-402c-b739-6f8a5bd22ba6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:48.464879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c492eb-b5f3-402c-b739-6f8a5bd22ba6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:48.476199 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1064/2124] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:48.481281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ea680-271a-426c-863b-5d960a863beb admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:48.494217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446ea680-271a-426c-863b-5d960a863beb admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:48.499946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5129ff-9b29-4bf6-86a2-3625c55c3d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:48.501004 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5129ff-9b29-4bf6-86a2-3625c55c3d84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:48.501004 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5129ff-9b29-4bf6-86a2-3625c55c3d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:48.501226 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5129ff-9b29-4bf6-86a2-3625c55c3d84 admin admin] PATH_INFO: `/v3/projects/db718e30a3d54deaab20dd3a17279fe9/users/49c9f15edb16431dbd39494df0d47dd5/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:48.512122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:48.520505 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5129ff-9b29-4bf6-86a2-3625c55c3d84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db718e30a3d54deaab20dd3a17279fe9, user_id=49c9f15edb16431dbd39494df0d47dd5, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:48.523150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5129ff-9b29-4bf6-86a2-3625c55c3d84 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:48.538620 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1061/2125] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:48 2022] PUT /identity/v3/projects/db718e30a3d54deaab20dd3a17279fe9/users/49c9f15edb16431dbd39494df0d47dd5/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:48.549559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc78cd0-ff93-47d5-86d0-9fdf1f984fbc None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:48.549846 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc78cd0-ff93-47d5-86d0-9fdf1f984fbc None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:48.550120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc78cd0-ff93-47d5-86d0-9fdf1f984fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:48.579503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-1bc78cd0-ff93-47d5-86d0-9fdf1f984fbc None None] MFA Rules not processed for user `49c9f15edb16431dbd39494df0d47dd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:48.580836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1bc78cd0-ff93-47d5-86d0-9fdf1f984fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:48.647105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1065/2126] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:13:48.658011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5129ff-9b29-4bf6-86a2-3625c55c3d84 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:48.659765 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1f5129ff-9b29-4bf6-86a2-3625c55c3d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:48.715755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5129ff-9b29-4bf6-86a2-3625c55c3d84 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:48.719385 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5802d7-b069-4f75-b8b7-dc2a55a2ffc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:48.719981 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5802d7-b069-4f75-b8b7-dc2a55a2ffc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:48.720281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5802d7-b069-4f75-b8b7-dc2a55a2ffc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:48.720548 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5802d7-b069-4f75-b8b7-dc2a55a2ffc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:48.729292 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5802d7-b069-4f75-b8b7-dc2a55a2ffc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:48.730644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5802d7-b069-4f75-b8b7-dc2a55a2ffc2 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:48.739521 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1062/2127] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:53.181275 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc78cd0-ff93-47d5-86d0-9fdf1f984fbc None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:53.182854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1bc78cd0-ff93-47d5-86d0-9fdf1f984fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:53.250061 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:53.254439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc78cd0-ff93-47d5-86d0-9fdf1f984fbc None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:53.259915 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe686e-b8ee-4db2-bb17-e7a111b5aebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:53.261063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe686e-b8ee-4db2-bb17-e7a111b5aebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:53.261374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe686e-b8ee-4db2-bb17-e7a111b5aebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:53.261648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafe686e-b8ee-4db2-bb17-e7a111b5aebc admin admin] PATH_INFO: `/v3/users/9566c18a8de14ce1923839669c211a8e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:53.262770 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe686e-b8ee-4db2-bb17-e7a111b5aebc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9566c18a8de14ce1923839669c211a8e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:53.264675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafe686e-b8ee-4db2-bb17-e7a111b5aebc admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:53.309990 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dafe686e-b8ee-4db2-bb17-e7a111b5aebc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9566c18a8de14ce1923839669c211a8e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:53.317137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dafe686e-b8ee-4db2-bb17-e7a111b5aebc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9566c18a8de14ce1923839669c211a8e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:53.325494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dafe686e-b8ee-4db2-bb17-e7a111b5aebc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9566c18a8de14ce1923839669c211a8e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:53.326112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dafe686e-b8ee-4db2-bb17-e7a111b5aebc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9566c18a8de14ce1923839669c211a8e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:53.340735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dafe686e-b8ee-4db2-bb17-e7a111b5aebc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9566c18a8de14ce1923839669c211a8e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:53.346599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1066/2128] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:53 2022] DELETE /identity/v3/users/9566c18a8de14ce1923839669c211a8e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:53.350951 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4718b-4cd4-4174-8f2c-903657232015 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:53.351437 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4718b-4cd4-4174-8f2c-903657232015 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:53.351654 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd4718b-4cd4-4174-8f2c-903657232015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:53.370916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4bd4718b-4cd4-4174-8f2c-903657232015 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:53.372250 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4bd4718b-4cd4-4174-8f2c-903657232015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:53.449376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1063/2129] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:53.459230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe686e-b8ee-4db2-bb17-e7a111b5aebc admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:53.461350 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-dafe686e-b8ee-4db2-bb17-e7a111b5aebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:53.521429 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafe686e-b8ee-4db2-bb17-e7a111b5aebc admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:53.525066 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09551630-5e39-43c8-afc3-c648b0ad517e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:53.525702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09551630-5e39-43c8-afc3-c648b0ad517e service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:53.525997 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09551630-5e39-43c8-afc3-c648b0ad517e service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:53.526277 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09551630-5e39-43c8-afc3-c648b0ad517e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:53.536318 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09551630-5e39-43c8-afc3-c648b0ad517e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:53.537908 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09551630-5e39-43c8-afc3-c648b0ad517e service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:53.547502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1067/2130] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:54.169256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4718b-4cd4-4174-8f2c-903657232015 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:54.170827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4bd4718b-4cd4-4174-8f2c-903657232015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:54.183295 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd4718b-4cd4-4174-8f2c-903657232015 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:54.188478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24694b89-6426-4c60-bab3-9f405499f31b None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:54.188973 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:54.189095 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24694b89-6426-4c60-bab3-9f405499f31b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:54.189332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24694b89-6426-4c60-bab3-9f405499f31b None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:54.194956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:54.194956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:54.194956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb admin admin] PATH_INFO: `/v3/projects/fa54ff36fd244c5bbb42a6db95f92d1b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:54.195280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1068/2131] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 19:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:13:54.195375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa54ff36fd244c5bbb42a6db95f92d1b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:54.195375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:54.228363 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcc2a6-e744-4a76-8646-f10ea40acba9 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:54.228649 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcc2a6-e744-4a76-8646-f10ea40acba9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:54.228975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbcc2a6-e744-4a76-8646-f10ea40acba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:54.278529 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-bdbcc2a6-e744-4a76-8646-f10ea40acba9 None None] MFA Rules not processed for user `74f1846e395e4d5483fcbb41eb28a77d`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:54.279812 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bdbcc2a6-e744-4a76-8646-f10ea40acba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:54.282843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa54ff36fd244c5bbb42a6db95f92d1b', 'request_id': 'req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:54.295616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa54ff36fd244c5bbb42a6db95f92d1b', 'request_id': 'req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:54.302880 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa54ff36fd244c5bbb42a6db95f92d1b', 'request_id': 'req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:54.320541 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb admin admin] The token cache is being invalidate because project fa54ff36fd244c5bbb42a6db95f92d1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:54.320849 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:54.322111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:54.324500 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1064/2132] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:54 2022] DELETE /identity/v3/projects/fa54ff36fd244c5bbb42a6db95f92d1b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:54.328318 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:54.330317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:54.349483 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:54.373022 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1069/2133] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 19:13:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:13:54.384555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbcc2a6-e744-4a76-8646-f10ea40acba9 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:54.386051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bdbcc2a6-e744-4a76-8646-f10ea40acba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:54.399714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbcc2a6-e744-4a76-8646-f10ea40acba9 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:54.405836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b4c3f-e45b-4ce6-b9b7-2aad6dfef501 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:54.406461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b4c3f-e45b-4ce6-b9b7-2aad6dfef501 service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:54.406735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b4c3f-e45b-4ce6-b9b7-2aad6dfef501 service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:54.407013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b4c3f-e45b-4ce6-b9b7-2aad6dfef501 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:54.416215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b4c3f-e45b-4ce6-b9b7-2aad6dfef501 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:54.417603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60ba2e-a9c6-42c1-98d8-8a44af018eeb admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:54.418375 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b4c3f-e45b-4ce6-b9b7-2aad6dfef501 service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:54.423025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183e389-57ad-407b-be8d-382d4144a345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:54.423494 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183e389-57ad-407b-be8d-382d4144a345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:54.423777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183e389-57ad-407b-be8d-382d4144a345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:54.424047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183e389-57ad-407b-be8d-382d4144a345 admin admin] PATH_INFO: `/v3/users/e9464ccb65f3421489c604118a9e4d45` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:54.425947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183e389-57ad-407b-be8d-382d4144a345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9464ccb65f3421489c604118a9e4d45)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:54.427976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183e389-57ad-407b-be8d-382d4144a345 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:54.434281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1070/2134] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:54.491890 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6183e389-57ad-407b-be8d-382d4144a345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9464ccb65f3421489c604118a9e4d45'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:54.499800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6183e389-57ad-407b-be8d-382d4144a345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9464ccb65f3421489c604118a9e4d45'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:54.513843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6183e389-57ad-407b-be8d-382d4144a345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9464ccb65f3421489c604118a9e4d45'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:54.514663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6183e389-57ad-407b-be8d-382d4144a345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9464ccb65f3421489c604118a9e4d45'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:54.515280 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6183e389-57ad-407b-be8d-382d4144a345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9464ccb65f3421489c604118a9e4d45'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:54.540564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1065/2135] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:54 2022] DELETE /identity/v3/users/e9464ccb65f3421489c604118a9e4d45 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:54.840518 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b4c3f-e45b-4ce6-b9b7-2aad6dfef501 service placement] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:54.847257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:54.860917 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b4c3f-e45b-4ce6-b9b7-2aad6dfef501 service placement] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:54.920950 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766565f-90db-468e-a136-9a72f0a5e11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:54.921613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766565f-90db-468e-a136-9a72f0a5e11c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:54.921898 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766565f-90db-468e-a136-9a72f0a5e11c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:54.922172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766565f-90db-468e-a136-9a72f0a5e11c admin admin] PATH_INFO: `/v3/projects/bb9e349fd77e4d9aa3844bdf6f1108c8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:54.923438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766565f-90db-468e-a136-9a72f0a5e11c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb9e349fd77e4d9aa3844bdf6f1108c8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:54.925358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766565f-90db-468e-a136-9a72f0a5e11c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:54.978184 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7766565f-90db-468e-a136-9a72f0a5e11c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb9e349fd77e4d9aa3844bdf6f1108c8', 'request_id': 'req-7766565f-90db-468e-a136-9a72f0a5e11c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:54.987399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7766565f-90db-468e-a136-9a72f0a5e11c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb9e349fd77e4d9aa3844bdf6f1108c8', 'request_id': 'req-7766565f-90db-468e-a136-9a72f0a5e11c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:54.993304 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7766565f-90db-468e-a136-9a72f0a5e11c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb9e349fd77e4d9aa3844bdf6f1108c8', 'request_id': 'req-7766565f-90db-468e-a136-9a72f0a5e11c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.006098 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7766565f-90db-468e-a136-9a72f0a5e11c admin admin] The token cache is being invalidate because project bb9e349fd77e4d9aa3844bdf6f1108c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:55.006403 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7766565f-90db-468e-a136-9a72f0a5e11c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.007124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7766565f-90db-468e-a136-9a72f0a5e11c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.008716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1071/2136] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:54 2022] DELETE /identity/v3/projects/bb9e349fd77e4d9aa3844bdf6f1108c8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:55.014323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7559eba-694c-4ce8-8df1-15567baddcaa None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:55.014612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7559eba-694c-4ce8-8df1-15567baddcaa None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:55.014884 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7559eba-694c-4ce8-8df1-15567baddcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:55.034993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f7559eba-694c-4ce8-8df1-15567baddcaa None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:55.036191 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f7559eba-694c-4ce8-8df1-15567baddcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:55.101831 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8de8d0-5874-4712-96be-468860310c8b None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:55.102159 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8de8d0-5874-4712-96be-468860310c8b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:55.102368 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8de8d0-5874-4712-96be-468860310c8b None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:55.102790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1066/2137] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:55.104023 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1072/2138] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 19:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:13:55.111495 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a112c8-e1f9-47a3-a13d-e8079d29aae8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:55.111776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a112c8-e1f9-47a3-a13d-e8079d29aae8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:55.112051 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a112c8-e1f9-47a3-a13d-e8079d29aae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:55.132210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-89a112c8-e1f9-47a3-a13d-e8079d29aae8 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:55.133451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-89a112c8-e1f9-47a3-a13d-e8079d29aae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:55.143656 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35605086-d93f-40ce-b535-e04f33eff43d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:55.143971 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35605086-d93f-40ce-b535-e04f33eff43d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:55.144221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35605086-d93f-40ce-b535-e04f33eff43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:55.147742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1067/2139] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:55.151848 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a112c8-e1f9-47a3-a13d-e8079d29aae8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:55.161473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a112c8-e1f9-47a3-a13d-e8079d29aae8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:55.167998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736d33a-263d-49ed-9d71-8ef3d8cd6d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:55.168746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736d33a-263d-49ed-9d71-8ef3d8cd6d52 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:55.169456 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736d33a-263d-49ed-9d71-8ef3d8cd6d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:55.169557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b736d33a-263d-49ed-9d71-8ef3d8cd6d52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:55.170177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736d33a-263d-49ed-9d71-8ef3d8cd6d52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:13:55.170596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736d33a-263d-49ed-9d71-8ef3d8cd6d52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:55.172354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b736d33a-263d-49ed-9d71-8ef3d8cd6d52 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:55.176111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-35605086-d93f-40ce-b535-e04f33eff43d None None] MFA Rules not processed for user `74f1846e395e4d5483fcbb41eb28a77d`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:55.178123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-35605086-d93f-40ce-b535-e04f33eff43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:55.193358 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1068/2140] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:13:55.195023 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736d33a-263d-49ed-9d71-8ef3d8cd6d52 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:55.200916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b736d33a-263d-49ed-9d71-8ef3d8cd6d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:55.214970 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b736d33a-263d-49ed-9d71-8ef3d8cd6d52 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:55.220229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfb7a3-7bda-416c-8c01-b4c4975a2625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:55.221112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfb7a3-7bda-416c-8c01-b4c4975a2625 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:55.221424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfb7a3-7bda-416c-8c01-b4c4975a2625 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:55.221697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cadfb7a3-7bda-416c-8c01-b4c4975a2625 admin admin] PATH_INFO: `/v3/users/f617dc8aec3a47b9bab93c5776cea215` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:55.222836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadfb7a3-7bda-416c-8c01-b4c4975a2625 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f617dc8aec3a47b9bab93c5776cea215)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:55.224740 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cadfb7a3-7bda-416c-8c01-b4c4975a2625 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:55.230685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:55.262994 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1073/2141] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 19:13:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:13:55.264138 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35605086-d93f-40ce-b535-e04f33eff43d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:55.275668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35605086-d93f-40ce-b535-e04f33eff43d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:55.280556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b1319-cece-4bdd-a052-a4a9f39d5053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:55.281379 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b1319-cece-4bdd-a052-a4a9f39d5053 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:55.281654 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b1319-cece-4bdd-a052-a4a9f39d5053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:55.281922 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b1319-cece-4bdd-a052-a4a9f39d5053 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:55.282541 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b1319-cece-4bdd-a052-a4a9f39d5053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:55.284417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351b1319-cece-4bdd-a052-a4a9f39d5053 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:55.286872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-cadfb7a3-7bda-416c-8c01-b4c4975a2625 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f617dc8aec3a47b9bab93c5776cea215'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.293792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-cadfb7a3-7bda-416c-8c01-b4c4975a2625 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f617dc8aec3a47b9bab93c5776cea215'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.306634 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-cadfb7a3-7bda-416c-8c01-b4c4975a2625 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f617dc8aec3a47b9bab93c5776cea215'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.307792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-cadfb7a3-7bda-416c-8c01-b4c4975a2625 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f617dc8aec3a47b9bab93c5776cea215'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.308442 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-cadfb7a3-7bda-416c-8c01-b4c4975a2625 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f617dc8aec3a47b9bab93c5776cea215'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.314084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1074/2142] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:55 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:55.315107 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b1319-cece-4bdd-a052-a4a9f39d5053 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:55.316682 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-351b1319-cece-4bdd-a052-a4a9f39d5053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:55.325487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1069/2143] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:55 2022] DELETE /identity/v3/users/f617dc8aec3a47b9bab93c5776cea215 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:55.325487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfb7a3-7bda-416c-8c01-b4c4975a2625 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:55.325487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cadfb7a3-7bda-416c-8c01-b4c4975a2625 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:55.330865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:55.398413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cadfb7a3-7bda-416c-8c01-b4c4975a2625 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:55.403194 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c7ef2-e9dd-42fd-b5a3-b5895072795d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:55.403872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c7ef2-e9dd-42fd-b5a3-b5895072795d admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:55.404221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c7ef2-e9dd-42fd-b5a3-b5895072795d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:55.404592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c7ef2-e9dd-42fd-b5a3-b5895072795d admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:55.405304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c7ef2-e9dd-42fd-b5a3-b5895072795d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:55.407268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c7ef2-e9dd-42fd-b5a3-b5895072795d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:55.410476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b1319-cece-4bdd-a052-a4a9f39d5053 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:55.432871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1070/2144] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:55 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:55.436784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c7ef2-e9dd-42fd-b5a3-b5895072795d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:55.447753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c7ef2-e9dd-42fd-b5a3-b5895072795d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:55.454539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6141a-0053-49c0-bab8-08f9062fdc34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:55.455142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6141a-0053-49c0-bab8-08f9062fdc34 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:55.455412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6141a-0053-49c0-bab8-08f9062fdc34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:55.455675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6141a-0053-49c0-bab8-08f9062fdc34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:55.456295 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6141a-0053-49c0-bab8-08f9062fdc34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:55.458064 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6141a-0053-49c0-bab8-08f9062fdc34 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:55.469969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1071/2145] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:13:55.473780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6141a-0053-49c0-bab8-08f9062fdc34 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:55.481583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa4398-29e0-4d72-9508-03896d338f97 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:55.482178 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa4398-29e0-4d72-9508-03896d338f97 service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:55.482723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa4398-29e0-4d72-9508-03896d338f97 service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:55.482998 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daa4398-29e0-4d72-9508-03896d338f97 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:55.485229 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1daa4398-29e0-4d72-9508-03896d338f97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:55.487326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6141a-0053-49c0-bab8-08f9062fdc34 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:55.492284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e686d7f-a732-416f-9782-f2b316c9a2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:55.493035 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e686d7f-a732-416f-9782-f2b316c9a2a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:55.493407 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e686d7f-a732-416f-9782-f2b316c9a2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:55.493686 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e686d7f-a732-416f-9782-f2b316c9a2a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:55.494311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e686d7f-a732-416f-9782-f2b316c9a2a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:55.496472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e686d7f-a732-416f-9782-f2b316c9a2a1 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:55.499456 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa4398-29e0-4d72-9508-03896d338f97 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:55.501014 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daa4398-29e0-4d72-9508-03896d338f97 service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:55.511770 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1075/2146] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:55.513362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1072/2147] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:13:55.514240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa4398-29e0-4d72-9508-03896d338f97 service placement] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:55.515911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e686d7f-a732-416f-9782-f2b316c9a2a1 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:55.516106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1daa4398-29e0-4d72-9508-03896d338f97 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:55.526950 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e686d7f-a732-416f-9782-f2b316c9a2a1 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:55.530850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daa4398-29e0-4d72-9508-03896d338f97 service placement] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:55.532689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2adaa2-dff6-49d0-b860-393d74822c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:55.533392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2adaa2-dff6-49d0-b860-393d74822c80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:55.533666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2adaa2-dff6-49d0-b860-393d74822c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:55.533934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2adaa2-dff6-49d0-b860-393d74822c80 admin admin] PATH_INFO: `/v3/projects/426e58f469324233b3514e866529eafa/users/e33c120ef02f4f6e8bea53107d90034b/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:55.538051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0f202-0b05-44c4-ab12-ee1d41242126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:55.538888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0f202-0b05-44c4-ab12-ee1d41242126 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:55.539172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0f202-0b05-44c4-ab12-ee1d41242126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:55.539445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd0f202-0b05-44c4-ab12-ee1d41242126 admin admin] PATH_INFO: `/v3/projects/0456da408bab448fbd6b16a09f7badd9` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:55.540601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0f202-0b05-44c4-ab12-ee1d41242126 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0456da408bab448fbd6b16a09f7badd9)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:55.543410 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd0f202-0b05-44c4-ab12-ee1d41242126 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:55.559988 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2adaa2-dff6-49d0-b860-393d74822c80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=426e58f469324233b3514e866529eafa, user_id=e33c120ef02f4f6e8bea53107d90034b, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:55.563563 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2adaa2-dff6-49d0-b860-393d74822c80 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:55.586040 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1073/2148] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:55 2022] PUT /identity/v3/projects/426e58f469324233b3514e866529eafa/users/e33c120ef02f4f6e8bea53107d90034b/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:55.587480 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db454476-a49b-41b4-8d47-34d1fabfb7f7 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:55.587767 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db454476-a49b-41b4-8d47-34d1fabfb7f7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:55.588044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db454476-a49b-41b4-8d47-34d1fabfb7f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:55.608286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-db454476-a49b-41b4-8d47-34d1fabfb7f7 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:55.609524 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-db454476-a49b-41b4-8d47-34d1fabfb7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:55.624694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1cd0f202-0b05-44c4-ab12-ee1d41242126 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0456da408bab448fbd6b16a09f7badd9', 'request_id': 'req-1cd0f202-0b05-44c4-ab12-ee1d41242126'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.630956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1cd0f202-0b05-44c4-ab12-ee1d41242126 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0456da408bab448fbd6b16a09f7badd9', 'request_id': 'req-1cd0f202-0b05-44c4-ab12-ee1d41242126'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.635343 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:55.637049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1cd0f202-0b05-44c4-ab12-ee1d41242126 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0456da408bab448fbd6b16a09f7badd9', 'request_id': 'req-1cd0f202-0b05-44c4-ab12-ee1d41242126'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.651160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1cd0f202-0b05-44c4-ab12-ee1d41242126 admin admin] The token cache is being invalidate because project 0456da408bab448fbd6b16a09f7badd9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:55.651449 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1cd0f202-0b05-44c4-ab12-ee1d41242126 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.652100 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1cd0f202-0b05-44c4-ab12-ee1d41242126 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.653871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1076/2149] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:55 2022] DELETE /identity/v3/projects/0456da408bab448fbd6b16a09f7badd9 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:55.657694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0f202-0b05-44c4-ab12-ee1d41242126 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:55.659026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1cd0f202-0b05-44c4-ab12-ee1d41242126 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:55.676196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1074/2150] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:13:55.686155 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:55.686396 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db454476-a49b-41b4-8d47-34d1fabfb7f7 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:55.687817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-db454476-a49b-41b4-8d47-34d1fabfb7f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:55.727569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd0f202-0b05-44c4-ab12-ee1d41242126 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:55.733176 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f046a-f6d0-4c7f-8aae-9ec08cfe4eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:55.733840 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f046a-f6d0-4c7f-8aae-9ec08cfe4eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:55.734134 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f046a-f6d0-4c7f-8aae-9ec08cfe4eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:55.734397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f046a-f6d0-4c7f-8aae-9ec08cfe4eb6 admin admin] PATH_INFO: `/v3/users/e601e3391db64336b8b8af3010c93bd4` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:55.735541 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f046a-f6d0-4c7f-8aae-9ec08cfe4eb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e601e3391db64336b8b8af3010c93bd4)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:55.737484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f046a-f6d0-4c7f-8aae-9ec08cfe4eb6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:55.745842 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db454476-a49b-41b4-8d47-34d1fabfb7f7 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:55.750093 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735a30a-188c-47fb-abe4-30746d20fcd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:55.750769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735a30a-188c-47fb-abe4-30746d20fcd5 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:55.751113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735a30a-188c-47fb-abe4-30746d20fcd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:55.751450 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735a30a-188c-47fb-abe4-30746d20fcd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:55.765710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735a30a-188c-47fb-abe4-30746d20fcd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:55.767718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735a30a-188c-47fb-abe4-30746d20fcd5 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:55.779864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1075/2151] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:55.801470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f62f046a-f6d0-4c7f-8aae-9ec08cfe4eb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e601e3391db64336b8b8af3010c93bd4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.812243 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f62f046a-f6d0-4c7f-8aae-9ec08cfe4eb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e601e3391db64336b8b8af3010c93bd4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.823826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f62f046a-f6d0-4c7f-8aae-9ec08cfe4eb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e601e3391db64336b8b8af3010c93bd4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.824578 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f62f046a-f6d0-4c7f-8aae-9ec08cfe4eb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e601e3391db64336b8b8af3010c93bd4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.852347 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f62f046a-f6d0-4c7f-8aae-9ec08cfe4eb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e601e3391db64336b8b8af3010c93bd4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:55.860437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1077/2152] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:55 2022] DELETE /identity/v3/users/e601e3391db64336b8b8af3010c93bd4 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:13:56.071652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735a30a-188c-47fb-abe4-30746d20fcd5 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:56.086977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735a30a-188c-47fb-abe4-30746d20fcd5 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:56.166136 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b61394e-c08f-4a47-a0e2-7d7419d54a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:56.167009 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61394e-c08f-4a47-a0e2-7d7419d54a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:56.167700 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61394e-c08f-4a47-a0e2-7d7419d54a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:56.167700 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b61394e-c08f-4a47-a0e2-7d7419d54a51 admin admin] PATH_INFO: `/v3/projects/d08e4234cb56400696801d46990f33b2` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:56.169008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61394e-c08f-4a47-a0e2-7d7419d54a51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d08e4234cb56400696801d46990f33b2)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:56.171451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b61394e-c08f-4a47-a0e2-7d7419d54a51 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:56.242730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7b61394e-c08f-4a47-a0e2-7d7419d54a51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd08e4234cb56400696801d46990f33b2', 'request_id': 'req-7b61394e-c08f-4a47-a0e2-7d7419d54a51'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:56.253534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7b61394e-c08f-4a47-a0e2-7d7419d54a51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd08e4234cb56400696801d46990f33b2', 'request_id': 'req-7b61394e-c08f-4a47-a0e2-7d7419d54a51'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:56.259810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7b61394e-c08f-4a47-a0e2-7d7419d54a51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd08e4234cb56400696801d46990f33b2', 'request_id': 'req-7b61394e-c08f-4a47-a0e2-7d7419d54a51'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:56.271826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7b61394e-c08f-4a47-a0e2-7d7419d54a51 admin admin] The token cache is being invalidate because project d08e4234cb56400696801d46990f33b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:56.272199 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7b61394e-c08f-4a47-a0e2-7d7419d54a51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:56.272924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7b61394e-c08f-4a47-a0e2-7d7419d54a51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:56.278970 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1076/2153] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:56 2022] DELETE /identity/v3/projects/d08e4234cb56400696801d46990f33b2 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:56.280983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4cce8-2a90-4d56-b6a7-9202a0c74e23 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:56.281499 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4cce8-2a90-4d56-b6a7-9202a0c74e23 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:56.281609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe4cce8-2a90-4d56-b6a7-9202a0c74e23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:56.302214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9fe4cce8-2a90-4d56-b6a7-9202a0c74e23 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:56.303427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9fe4cce8-2a90-4d56-b6a7-9202a0c74e23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:56.370496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1078/2154] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:56.379682 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffe62b-6a39-4452-9683-e09ad806b244 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:56.379968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffe62b-6a39-4452-9683-e09ad806b244 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:56.380704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ffe62b-6a39-4452-9683-e09ad806b244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:56.389078 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:56.403933 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-93ffe62b-6a39-4452-9683-e09ad806b244 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:56.405160 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-93ffe62b-6a39-4452-9683-e09ad806b244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:56.413734 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1077/2155] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:56.418653 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4cce8-2a90-4d56-b6a7-9202a0c74e23 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:56.431258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe4cce8-2a90-4d56-b6a7-9202a0c74e23 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:56.435973 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb60400-9137-45fb-ad33-33dbc886d26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:56.436636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb60400-9137-45fb-ad33-33dbc886d26f admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:56.436955 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb60400-9137-45fb-ad33-33dbc886d26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:56.437282 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb60400-9137-45fb-ad33-33dbc886d26f admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:56.437993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb60400-9137-45fb-ad33-33dbc886d26f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:13:56.438331 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb60400-9137-45fb-ad33-33dbc886d26f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:56.440167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb60400-9137-45fb-ad33-33dbc886d26f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:56.453699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f7b5d-568a-4233-9e9b-d161ed835436 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:56.454217 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1079/2156] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:13:56.454323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f7b5d-568a-4233-9e9b-d161ed835436 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:56.454323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2f7b5d-568a-4233-9e9b-d161ed835436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:56.457919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb60400-9137-45fb-ad33-33dbc886d26f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:56.469856 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb60400-9137-45fb-ad33-33dbc886d26f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:56.474945 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f800f0-6dfc-449f-b33e-aa7adb9d3a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:56.475556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f800f0-6dfc-449f-b33e-aa7adb9d3a2e admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:56.475874 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f800f0-6dfc-449f-b33e-aa7adb9d3a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:56.476271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f800f0-6dfc-449f-b33e-aa7adb9d3a2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:56.476961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f800f0-6dfc-449f-b33e-aa7adb9d3a2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:56.480052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f800f0-6dfc-449f-b33e-aa7adb9d3a2e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:56.489299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-fb2f7b5d-568a-4233-9e9b-d161ed835436 None None] MFA Rules not processed for user `e33c120ef02f4f6e8bea53107d90034b`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:56.491070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-fb2f7b5d-568a-4233-9e9b-d161ed835436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:56.514408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1080/2157] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:56 2022] POST /identity/v3/projects => generated 361 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:13:56.516328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f800f0-6dfc-449f-b33e-aa7adb9d3a2e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:56.525240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:56.569502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1078/2158] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:13:56.581507 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f7b5d-568a-4233-9e9b-d161ed835436 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:56.582925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-fb2f7b5d-568a-4233-9e9b-d161ed835436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:56.591192 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f800f0-6dfc-449f-b33e-aa7adb9d3a2e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:56.597154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040f126-2bf9-4111-ae3d-5a6496f4fc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:56.597972 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040f126-2bf9-4111-ae3d-5a6496f4fc51 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:56.598257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040f126-2bf9-4111-ae3d-5a6496f4fc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:56.598575 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040f126-2bf9-4111-ae3d-5a6496f4fc51 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:56.599210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040f126-2bf9-4111-ae3d-5a6496f4fc51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:56.601283 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040f126-2bf9-4111-ae3d-5a6496f4fc51 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:56.628215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1081/2159] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:56 2022] POST /identity/v3/users => generated 281 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:56.632312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040f126-2bf9-4111-ae3d-5a6496f4fc51 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:56.643965 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040f126-2bf9-4111-ae3d-5a6496f4fc51 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:56.647418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2f7b5d-568a-4233-9e9b-d161ed835436 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:56.649626 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4cef9-4e08-4e77-b238-fe823319decd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:56.650302 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4cef9-4e08-4e77-b238-fe823319decd admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:56.651008 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4cef9-4e08-4e77-b238-fe823319decd admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:56.651295 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4cef9-4e08-4e77-b238-fe823319decd admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:56.652012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4cef9-4e08-4e77-b238-fe823319decd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:56.653707 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4cef9-4e08-4e77-b238-fe823319decd admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:56.664513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1082/2160] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:13:56.668290 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4cef9-4e08-4e77-b238-fe823319decd admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:56.678177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4cef9-4e08-4e77-b238-fe823319decd admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:56.683845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693521e-01e4-488d-b1df-502c3bf38cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:56.684432 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693521e-01e4-488d-b1df-502c3bf38cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:56.684710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693521e-01e4-488d-b1df-502c3bf38cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:56.684980 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b693521e-01e4-488d-b1df-502c3bf38cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:56.685644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b693521e-01e4-488d-b1df-502c3bf38cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:56.687223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b693521e-01e4-488d-b1df-502c3bf38cd4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:56.697891 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1083/2161] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:13:56.702283 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693521e-01e4-488d-b1df-502c3bf38cd4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:56.702702 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ddce1-2533-49e5-a2fb-730628ed7f9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:56.703387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ddce1-2533-49e5-a2fb-730628ed7f9b service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:56.703742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ddce1-2533-49e5-a2fb-730628ed7f9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:56.704086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ddce1-2533-49e5-a2fb-730628ed7f9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:56.712191 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b693521e-01e4-488d-b1df-502c3bf38cd4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:56.716913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcf8824-8c23-4c66-9572-daa672086a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:56.717563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf8824-8c23-4c66-9572-daa672086a56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:56.717842 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf8824-8c23-4c66-9572-daa672086a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:56.718113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcf8824-8c23-4c66-9572-daa672086a56 admin admin] PATH_INFO: `/v3/projects/7765877b82a843f0b9d87a61c781001e/users/785f14cb38a749a1810cbf25c982174c/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:56.733116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf8824-8c23-4c66-9572-daa672086a56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7765877b82a843f0b9d87a61c781001e, user_id=785f14cb38a749a1810cbf25c982174c, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:56.735766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcf8824-8c23-4c66-9572-daa672086a56 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:56.749231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1084/2162] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:56 2022] PUT /identity/v3/projects/7765877b82a843f0b9d87a61c781001e/users/785f14cb38a749a1810cbf25c982174c/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:13:56.758687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:56.759191 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf32b0-cc96-430e-9144-84d890d91395 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:56.759473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf32b0-cc96-430e-9144-84d890d91395 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:56.759777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bf32b0-cc96-430e-9144-84d890d91395 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:56.761827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ddce1-2533-49e5-a2fb-730628ed7f9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:56.763860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ddce1-2533-49e5-a2fb-730628ed7f9b service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:56.773565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1079/2163] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:56 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:13:56.789229 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e5bf32b0-cc96-430e-9144-84d890d91395 None None] MFA Rules not processed for user `785f14cb38a749a1810cbf25c982174c`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:56.790309 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e5bf32b0-cc96-430e-9144-84d890d91395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:56.855397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1085/2164] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:56 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:57.204424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ddce1-2533-49e5-a2fb-730628ed7f9b service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:57.213282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ddce1-2533-49e5-a2fb-730628ed7f9b service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:57.263222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdec63-561a-4d8a-a5b1-175d1b494ea2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:57.263851 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdec63-561a-4d8a-a5b1-175d1b494ea2 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:57.264129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdec63-561a-4d8a-a5b1-175d1b494ea2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:57.264401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdec63-561a-4d8a-a5b1-175d1b494ea2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:57.272705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdec63-561a-4d8a-a5b1-175d1b494ea2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:57.274216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdec63-561a-4d8a-a5b1-175d1b494ea2 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:57.283773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1080/2165] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:57 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:57.299984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf32b0-cc96-430e-9144-84d890d91395 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:57.343104 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:57.365118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bf32b0-cc96-430e-9144-84d890d91395 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:57.369800 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915aaf9-1e82-47a4-97ff-559ce75ac97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:57.370459 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915aaf9-1e82-47a4-97ff-559ce75ac97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:57.370734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915aaf9-1e82-47a4-97ff-559ce75ac97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:57.371003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c915aaf9-1e82-47a4-97ff-559ce75ac97b admin admin] PATH_INFO: `/v3/users/785f14cb38a749a1810cbf25c982174c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:57.372117 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915aaf9-1e82-47a4-97ff-559ce75ac97b admin admin] RBAC: Authorizing `identity:delete_user(user_id=785f14cb38a749a1810cbf25c982174c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:57.374059 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c915aaf9-1e82-47a4-97ff-559ce75ac97b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:57.419843 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c915aaf9-1e82-47a4-97ff-559ce75ac97b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '785f14cb38a749a1810cbf25c982174c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:57.427114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c915aaf9-1e82-47a4-97ff-559ce75ac97b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '785f14cb38a749a1810cbf25c982174c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:57.433968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c915aaf9-1e82-47a4-97ff-559ce75ac97b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '785f14cb38a749a1810cbf25c982174c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:57.434829 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c915aaf9-1e82-47a4-97ff-559ce75ac97b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '785f14cb38a749a1810cbf25c982174c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:57.446795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c915aaf9-1e82-47a4-97ff-559ce75ac97b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '785f14cb38a749a1810cbf25c982174c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:57.452740 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1086/2166] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:13:57 2022] DELETE /identity/v3/users/785f14cb38a749a1810cbf25c982174c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:57.457156 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f387a7b-aa3e-4958-bd98-86b1b73e5b52 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:57.457644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f387a7b-aa3e-4958-bd98-86b1b73e5b52 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:57.457732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f387a7b-aa3e-4958-bd98-86b1b73e5b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:57.476834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6f387a7b-aa3e-4958-bd98-86b1b73e5b52 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:57.478078 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6f387a7b-aa3e-4958-bd98-86b1b73e5b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:57.535482 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1081/2167] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:57.545980 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915aaf9-1e82-47a4-97ff-559ce75ac97b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:57.547389 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c915aaf9-1e82-47a4-97ff-559ce75ac97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:57.603463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c915aaf9-1e82-47a4-97ff-559ce75ac97b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:57.608056 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec0989e-c8c0-4957-8fe9-21b7d087c271 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:57.608651 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0989e-c8c0-4957-8fe9-21b7d087c271 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:57.608923 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0989e-c8c0-4957-8fe9-21b7d087c271 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:57.609201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec0989e-c8c0-4957-8fe9-21b7d087c271 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:57.618293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec0989e-c8c0-4957-8fe9-21b7d087c271 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:57.620158 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec0989e-c8c0-4957-8fe9-21b7d087c271 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:57.629793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1087/2168] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:13:57.902273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f387a7b-aa3e-4958-bd98-86b1b73e5b52 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:57.903715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6f387a7b-aa3e-4958-bd98-86b1b73e5b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:57.916101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f387a7b-aa3e-4958-bd98-86b1b73e5b52 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:57.921398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cb718a-7db8-42d6-aa5c-c9c3113a5ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:57.922033 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb718a-7db8-42d6-aa5c-c9c3113a5ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:57.922308 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb718a-7db8-42d6-aa5c-c9c3113a5ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:57.922578 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cb718a-7db8-42d6-aa5c-c9c3113a5ce5 admin admin] PATH_INFO: `/v3/projects/7765877b82a843f0b9d87a61c781001e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:57.923787 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb718a-7db8-42d6-aa5c-c9c3113a5ce5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7765877b82a843f0b9d87a61c781001e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:57.925754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cb718a-7db8-42d6-aa5c-c9c3113a5ce5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:57.975522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-65cb718a-7db8-42d6-aa5c-c9c3113a5ce5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7765877b82a843f0b9d87a61c781001e', 'request_id': 'req-65cb718a-7db8-42d6-aa5c-c9c3113a5ce5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:57.983272 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-65cb718a-7db8-42d6-aa5c-c9c3113a5ce5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7765877b82a843f0b9d87a61c781001e', 'request_id': 'req-65cb718a-7db8-42d6-aa5c-c9c3113a5ce5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:57.989092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-65cb718a-7db8-42d6-aa5c-c9c3113a5ce5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7765877b82a843f0b9d87a61c781001e', 'request_id': 'req-65cb718a-7db8-42d6-aa5c-c9c3113a5ce5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:58.000758 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-65cb718a-7db8-42d6-aa5c-c9c3113a5ce5 admin admin] The token cache is being invalidate because project 7765877b82a843f0b9d87a61c781001e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:13:58.001042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-65cb718a-7db8-42d6-aa5c-c9c3113a5ce5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:58.001723 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-65cb718a-7db8-42d6-aa5c-c9c3113a5ce5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:13:58.003204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1082/2169] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:13:57 2022] DELETE /identity/v3/projects/7765877b82a843f0b9d87a61c781001e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:58.008144 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fdcb4a-b9fc-481f-8ae2-deaa5025d152 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:58.008426 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fdcb4a-b9fc-481f-8ae2-deaa5025d152 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:58.008698 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fdcb4a-b9fc-481f-8ae2-deaa5025d152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:58.028240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-17fdcb4a-b9fc-481f-8ae2-deaa5025d152 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:58.029474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-17fdcb4a-b9fc-481f-8ae2-deaa5025d152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:58.090768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1088/2170] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:58.098821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b09a1-2027-45d2-a3eb-3fc016568996 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:58.099126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b09a1-2027-45d2-a3eb-3fc016568996 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:58.099406 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3b09a1-2027-45d2-a3eb-3fc016568996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:58.120331 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9f3b09a1-2027-45d2-a3eb-3fc016568996 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:58.122228 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9f3b09a1-2027-45d2-a3eb-3fc016568996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:58.130287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1083/2171] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:58.134656 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fdcb4a-b9fc-481f-8ae2-deaa5025d152 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:58.140283 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:58.147245 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fdcb4a-b9fc-481f-8ae2-deaa5025d152 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:58.154959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900af6ea-e246-4328-b4e7-1a068fc3a414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:58.155569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900af6ea-e246-4328-b4e7-1a068fc3a414 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:58.155848 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900af6ea-e246-4328-b4e7-1a068fc3a414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:58.157668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900af6ea-e246-4328-b4e7-1a068fc3a414 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:58.157668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900af6ea-e246-4328-b4e7-1a068fc3a414 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:13:58.157668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900af6ea-e246-4328-b4e7-1a068fc3a414 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:58.158731 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900af6ea-e246-4328-b4e7-1a068fc3a414 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:58.175433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1089/2172] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:13:58.178059 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b09a1-2027-45d2-a3eb-3fc016568996 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:58.204223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3b09a1-2027-45d2-a3eb-3fc016568996 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:58.221901 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c982668-eee7-4441-92d1-3d1b987f7a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:58.222561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c982668-eee7-4441-92d1-3d1b987f7a7a admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:58.222843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c982668-eee7-4441-92d1-3d1b987f7a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:58.223114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c982668-eee7-4441-92d1-3d1b987f7a7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:58.223746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c982668-eee7-4441-92d1-3d1b987f7a7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:58.226799 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c982668-eee7-4441-92d1-3d1b987f7a7a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:58.244139 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:13:58.265720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1084/2173] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:13:58 2022] POST /identity/v3/projects => generated 375 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:58.270326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900af6ea-e246-4328-b4e7-1a068fc3a414 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:58.281327 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900af6ea-e246-4328-b4e7-1a068fc3a414 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:58.345999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112c953-8626-4049-adc0-2a54ccfc4b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:58.346597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112c953-8626-4049-adc0-2a54ccfc4b1b admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:58.346834 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112c953-8626-4049-adc0-2a54ccfc4b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:58.347069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1112c953-8626-4049-adc0-2a54ccfc4b1b admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:58.347695 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112c953-8626-4049-adc0-2a54ccfc4b1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:58.349762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1112c953-8626-4049-adc0-2a54ccfc4b1b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:58.373144 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1090/2174] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:13:58 2022] POST /identity/v3/users => generated 288 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:13:58.376752 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c982668-eee7-4441-92d1-3d1b987f7a7a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:58.388315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c982668-eee7-4441-92d1-3d1b987f7a7a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:58.393288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069a554-fe21-4be0-a327-a69837c85d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:58.393936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069a554-fe21-4be0-a327-a69837c85d7c admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:58.394214 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069a554-fe21-4be0-a327-a69837c85d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:58.394512 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a069a554-fe21-4be0-a327-a69837c85d7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:58.395187 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a069a554-fe21-4be0-a327-a69837c85d7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:58.398304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a069a554-fe21-4be0-a327-a69837c85d7c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:58.415817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1085/2175] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:58.422270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112c953-8626-4049-adc0-2a54ccfc4b1b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:58.433514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1112c953-8626-4049-adc0-2a54ccfc4b1b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:58.441579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df197d7c-103b-4e57-95b3-f7edf28f8570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:58.442393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df197d7c-103b-4e57-95b3-f7edf28f8570 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:58.442734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df197d7c-103b-4e57-95b3-f7edf28f8570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:58.442944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df197d7c-103b-4e57-95b3-f7edf28f8570 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:58.443711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df197d7c-103b-4e57-95b3-f7edf28f8570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:58.445455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df197d7c-103b-4e57-95b3-f7edf28f8570 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:58.457643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1091/2176] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:13:58.461997 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069a554-fe21-4be0-a327-a69837c85d7c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:58.472450 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a069a554-fe21-4be0-a327-a69837c85d7c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:58.477455 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8d17d-5b0a-4355-9458-efe6a6a670cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:58.478138 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8d17d-5b0a-4355-9458-efe6a6a670cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:58.478425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8d17d-5b0a-4355-9458-efe6a6a670cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:58.478656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe8d17d-5b0a-4355-9458-efe6a6a670cd admin admin] PATH_INFO: `/v3/projects/3d581d500fdb44e4a3845dfcecc7d498/users/79ee3ada54674744b9a3126930d97caa/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:58.497602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8d17d-5b0a-4355-9458-efe6a6a670cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d581d500fdb44e4a3845dfcecc7d498, user_id=79ee3ada54674744b9a3126930d97caa, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:58.500371 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe8d17d-5b0a-4355-9458-efe6a6a670cd admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:58.513831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1086/2177] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:13:58 2022] PUT /identity/v3/projects/3d581d500fdb44e4a3845dfcecc7d498/users/79ee3ada54674744b9a3126930d97caa/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:13:58.523889 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e0622-93ee-4dad-8ffd-1d7795c2d1a5 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:58.524172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e0622-93ee-4dad-8ffd-1d7795c2d1a5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:58.524442 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e0622-93ee-4dad-8ffd-1d7795c2d1a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:58.553690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-594e0622-93ee-4dad-8ffd-1d7795c2d1a5 None None] MFA Rules not processed for user `79ee3ada54674744b9a3126930d97caa`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:13:58.554895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-594e0622-93ee-4dad-8ffd-1d7795c2d1a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:58.624379 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1092/2178] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:13:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:13:58.635781 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8d17d-5b0a-4355-9458-efe6a6a670cd admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:13:58.637562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3fe8d17d-5b0a-4355-9458-efe6a6a670cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:13:58.694425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe8d17d-5b0a-4355-9458-efe6a6a670cd admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:13:58.698090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e2998-5411-4bd8-98c9-aea3e9f9a10b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:13:58.698674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e2998-5411-4bd8-98c9-aea3e9f9a10b service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:13:58.698946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e2998-5411-4bd8-98c9-aea3e9f9a10b service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:13:58.699209 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6e2998-5411-4bd8-98c9-aea3e9f9a10b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:13:58.707315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e2998-5411-4bd8-98c9-aea3e9f9a10b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:13:58.708765 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6e2998-5411-4bd8-98c9-aea3e9f9a10b service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:13:58.717725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1087/2179] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:13:58 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:14:00.451552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902700db-aaa3-447a-b390-2275f67e1cf9 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:00.452082 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902700db-aaa3-447a-b390-2275f67e1cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:00.452282 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902700db-aaa3-447a-b390-2275f67e1cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:00.471759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-902700db-aaa3-447a-b390-2275f67e1cf9 None None] MFA Rules not processed for user `e33c120ef02f4f6e8bea53107d90034b`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:00.472963 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-902700db-aaa3-447a-b390-2275f67e1cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:00.537083 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1093/2180] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:07.126679 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e2998-5411-4bd8-98c9-aea3e9f9a10b service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:07.128596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3e6e2998-5411-4bd8-98c9-aea3e9f9a10b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:07.200439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6e2998-5411-4bd8-98c9-aea3e9f9a10b service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:07.204968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3ef6b-5c5b-4611-810b-002fbbee87c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:07.205607 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3ef6b-5c5b-4611-810b-002fbbee87c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:07.205881 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3ef6b-5c5b-4611-810b-002fbbee87c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:07.206148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e3ef6b-5c5b-4611-810b-002fbbee87c3 admin admin] PATH_INFO: `/v3/users/ed922e856cad4be19467bc2a998d576f` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:07.208044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3ef6b-5c5b-4611-810b-002fbbee87c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed922e856cad4be19467bc2a998d576f)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:07.210914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e3ef6b-5c5b-4611-810b-002fbbee87c3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:07.258517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f1e3ef6b-5c5b-4611-810b-002fbbee87c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed922e856cad4be19467bc2a998d576f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:07.265300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f1e3ef6b-5c5b-4611-810b-002fbbee87c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed922e856cad4be19467bc2a998d576f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:07.270670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:INSERT=2,keystone:SELECT=90,keystone:DELETE=8 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:07.279879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f1e3ef6b-5c5b-4611-810b-002fbbee87c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed922e856cad4be19467bc2a998d576f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:07.280748 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f1e3ef6b-5c5b-4611-810b-002fbbee87c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed922e856cad4be19467bc2a998d576f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:07.281665 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f1e3ef6b-5c5b-4611-810b-002fbbee87c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed922e856cad4be19467bc2a998d576f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:07.295405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1088/2181] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:14:07 2022] DELETE /identity/v3/users/ed922e856cad4be19467bc2a998d576f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:07.300247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31d508-5131-458b-bdca-0851dcffef9d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:07.300819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31d508-5131-458b-bdca-0851dcffef9d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:07.300927 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f31d508-5131-458b-bdca-0851dcffef9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:07.308216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:07.321470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2f31d508-5131-458b-bdca-0851dcffef9d None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:07.322666 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2f31d508-5131-458b-bdca-0851dcffef9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:07.388099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1094/2182] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:14:07.397971 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3ef6b-5c5b-4611-810b-002fbbee87c3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:07.399539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f1e3ef6b-5c5b-4611-810b-002fbbee87c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:07.462615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e3ef6b-5c5b-4611-810b-002fbbee87c3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:07.467328 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69669be5-abea-4bcd-9b94-e3de2792aba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:07.468027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69669be5-abea-4bcd-9b94-e3de2792aba6 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:07.468331 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69669be5-abea-4bcd-9b94-e3de2792aba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:07.468708 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69669be5-abea-4bcd-9b94-e3de2792aba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:07.478397 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69669be5-abea-4bcd-9b94-e3de2792aba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:07.480063 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69669be5-abea-4bcd-9b94-e3de2792aba6 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:07.490600 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1089/2183] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:14:07.809062 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31d508-5131-458b-bdca-0851dcffef9d None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:07.810685 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2f31d508-5131-458b-bdca-0851dcffef9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:07.824263 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f31d508-5131-458b-bdca-0851dcffef9d None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:07.830596 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd80dd-3d13-440e-9d7c-19169c2abe68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:07.831239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd80dd-3d13-440e-9d7c-19169c2abe68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:07.831518 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd80dd-3d13-440e-9d7c-19169c2abe68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:07.831806 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd80dd-3d13-440e-9d7c-19169c2abe68 admin admin] PATH_INFO: `/v3/projects/b18fac49577a4056bf6c5bbfed781f11` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:07.832951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd80dd-3d13-440e-9d7c-19169c2abe68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b18fac49577a4056bf6c5bbfed781f11)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:07.844735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfd80dd-3d13-440e-9d7c-19169c2abe68 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:07.907353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebfd80dd-3d13-440e-9d7c-19169c2abe68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b18fac49577a4056bf6c5bbfed781f11', 'request_id': 'req-ebfd80dd-3d13-440e-9d7c-19169c2abe68'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:07.913797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebfd80dd-3d13-440e-9d7c-19169c2abe68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b18fac49577a4056bf6c5bbfed781f11', 'request_id': 'req-ebfd80dd-3d13-440e-9d7c-19169c2abe68'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:07.919861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebfd80dd-3d13-440e-9d7c-19169c2abe68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b18fac49577a4056bf6c5bbfed781f11', 'request_id': 'req-ebfd80dd-3d13-440e-9d7c-19169c2abe68'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:07.933578 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebfd80dd-3d13-440e-9d7c-19169c2abe68 admin admin] The token cache is being invalidate because project b18fac49577a4056bf6c5bbfed781f11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:14:07.933866 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebfd80dd-3d13-440e-9d7c-19169c2abe68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:07.934503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebfd80dd-3d13-440e-9d7c-19169c2abe68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:07.936000 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1095/2184] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:14:07 2022] DELETE /identity/v3/projects/b18fac49577a4056bf6c5bbfed781f11 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:07.940002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69669be5-abea-4bcd-9b94-e3de2792aba6 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:07.950276 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69669be5-abea-4bcd-9b94-e3de2792aba6 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:08.015389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b28e28-ed8d-4be9-bf74-6a6cb57c43ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:08.016027 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b28e28-ed8d-4be9-bf74-6a6cb57c43ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:08.016301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b28e28-ed8d-4be9-bf74-6a6cb57c43ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:08.016572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b28e28-ed8d-4be9-bf74-6a6cb57c43ce admin admin] PATH_INFO: `/v3/users/4abcec2a15fa42fe8ec5228523afa4e4` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:08.017717 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b28e28-ed8d-4be9-bf74-6a6cb57c43ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=4abcec2a15fa42fe8ec5228523afa4e4)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:08.019655 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b28e28-ed8d-4be9-bf74-6a6cb57c43ce admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:08.064129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d1b28e28-ed8d-4be9-bf74-6a6cb57c43ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4abcec2a15fa42fe8ec5228523afa4e4'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.070986 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d1b28e28-ed8d-4be9-bf74-6a6cb57c43ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4abcec2a15fa42fe8ec5228523afa4e4'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.083361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d1b28e28-ed8d-4be9-bf74-6a6cb57c43ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4abcec2a15fa42fe8ec5228523afa4e4'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.084025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d1b28e28-ed8d-4be9-bf74-6a6cb57c43ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4abcec2a15fa42fe8ec5228523afa4e4'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.084636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d1b28e28-ed8d-4be9-bf74-6a6cb57c43ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4abcec2a15fa42fe8ec5228523afa4e4'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.096860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1090/2185] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:14:07 2022] DELETE /identity/v3/users/4abcec2a15fa42fe8ec5228523afa4e4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:08.160467 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd80dd-3d13-440e-9d7c-19169c2abe68 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:08.162207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ebfd80dd-3d13-440e-9d7c-19169c2abe68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:08.176746 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd80dd-3d13-440e-9d7c-19169c2abe68 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:08.244182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1485b-4115-4b85-91ed-8ce275d7ae47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:08.244820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1485b-4115-4b85-91ed-8ce275d7ae47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:08.245170 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1485b-4115-4b85-91ed-8ce275d7ae47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:08.245964 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1485b-4115-4b85-91ed-8ce275d7ae47 admin admin] PATH_INFO: `/v3/users/5e7c7d232fa9470b868bcf60a8a35d88` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:08.247136 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1485b-4115-4b85-91ed-8ce275d7ae47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e7c7d232fa9470b868bcf60a8a35d88)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:08.249172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1485b-4115-4b85-91ed-8ce275d7ae47 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:08.279591 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:08.299778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-94a1485b-4115-4b85-91ed-8ce275d7ae47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e7c7d232fa9470b868bcf60a8a35d88'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.306845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-94a1485b-4115-4b85-91ed-8ce275d7ae47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e7c7d232fa9470b868bcf60a8a35d88'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.315851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-94a1485b-4115-4b85-91ed-8ce275d7ae47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e7c7d232fa9470b868bcf60a8a35d88'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.316553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-94a1485b-4115-4b85-91ed-8ce275d7ae47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e7c7d232fa9470b868bcf60a8a35d88'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.329454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-94a1485b-4115-4b85-91ed-8ce275d7ae47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e7c7d232fa9470b868bcf60a8a35d88'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.335375 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1096/2186] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:14:08 2022] DELETE /identity/v3/users/5e7c7d232fa9470b868bcf60a8a35d88 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:08.339879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e60f08-2223-4624-91f7-67ecd1c7c6fa None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:08.340349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e60f08-2223-4624-91f7-67ecd1c7c6fa None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:08.340441 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e60f08-2223-4624-91f7-67ecd1c7c6fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:08.347627 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:08.363238 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-94e60f08-2223-4624-91f7-67ecd1c7c6fa None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:08.364453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-94e60f08-2223-4624-91f7-67ecd1c7c6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:08.393653 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1485b-4115-4b85-91ed-8ce275d7ae47 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:08.395261 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-94a1485b-4115-4b85-91ed-8ce275d7ae47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:08.440141 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1091/2187] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:14:08.447657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e60f08-2223-4624-91f7-67ecd1c7c6fa None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:08.449236 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-94e60f08-2223-4624-91f7-67ecd1c7c6fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:08.478399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1485b-4115-4b85-91ed-8ce275d7ae47 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:08.483845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5414c89-75a4-4a28-a16f-a68b14be31c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:08.484474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5414c89-75a4-4a28-a16f-a68b14be31c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:08.484747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5414c89-75a4-4a28-a16f-a68b14be31c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:08.485020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5414c89-75a4-4a28-a16f-a68b14be31c1 admin admin] PATH_INFO: `/v3/projects/c8bb0277129c475e96b8daecd58a3386` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:08.486139 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5414c89-75a4-4a28-a16f-a68b14be31c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8bb0277129c475e96b8daecd58a3386)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:08.488056 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5414c89-75a4-4a28-a16f-a68b14be31c1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:08.511300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e60f08-2223-4624-91f7-67ecd1c7c6fa None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:08.515331 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601e0fd-a98a-4d8b-a5c2-cd8fca51ea5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:08.515944 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601e0fd-a98a-4d8b-a5c2-cd8fca51ea5e service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:08.516219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601e0fd-a98a-4d8b-a5c2-cd8fca51ea5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:08.516489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601e0fd-a98a-4d8b-a5c2-cd8fca51ea5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:08.527233 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601e0fd-a98a-4d8b-a5c2-cd8fca51ea5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:08.528898 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601e0fd-a98a-4d8b-a5c2-cd8fca51ea5e service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:08.539803 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1092/2188] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:14:08.552032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5414c89-75a4-4a28-a16f-a68b14be31c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8bb0277129c475e96b8daecd58a3386', 'request_id': 'req-f5414c89-75a4-4a28-a16f-a68b14be31c1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.558450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5414c89-75a4-4a28-a16f-a68b14be31c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8bb0277129c475e96b8daecd58a3386', 'request_id': 'req-f5414c89-75a4-4a28-a16f-a68b14be31c1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.564013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5414c89-75a4-4a28-a16f-a68b14be31c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8bb0277129c475e96b8daecd58a3386', 'request_id': 'req-f5414c89-75a4-4a28-a16f-a68b14be31c1'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.577893 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5414c89-75a4-4a28-a16f-a68b14be31c1 admin admin] The token cache is being invalidate because project c8bb0277129c475e96b8daecd58a3386 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:14:08.578231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5414c89-75a4-4a28-a16f-a68b14be31c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.579020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5414c89-75a4-4a28-a16f-a68b14be31c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.580661 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1097/2189] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:14:08 2022] DELETE /identity/v3/projects/c8bb0277129c475e96b8daecd58a3386 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:08.585702 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8ff22-97bc-48fa-863b-c3cf7931a8fa None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:08.585930 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8ff22-97bc-48fa-863b-c3cf7931a8fa None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:08.586300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc8ff22-97bc-48fa-863b-c3cf7931a8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:08.609271 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-1bc8ff22-97bc-48fa-863b-c3cf7931a8fa None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:08.610427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1bc8ff22-97bc-48fa-863b-c3cf7931a8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:08.672179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1093/2190] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:08.681580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed776da-10f7-4f32-a95d-a00b676fc7c0 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:08.681580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed776da-10f7-4f32-a95d-a00b676fc7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:08.681580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed776da-10f7-4f32-a95d-a00b676fc7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:08.706034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-eed776da-10f7-4f32-a95d-a00b676fc7c0 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:08.707423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-eed776da-10f7-4f32-a95d-a00b676fc7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:08.717429 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1098/2191] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:14:08.721791 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8ff22-97bc-48fa-863b-c3cf7931a8fa None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:08.732648 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc8ff22-97bc-48fa-863b-c3cf7931a8fa None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:08.737425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748a444-0039-4ce0-ade3-da67c9229853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:08.738028 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748a444-0039-4ce0-ade3-da67c9229853 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:08.738305 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748a444-0039-4ce0-ade3-da67c9229853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:08.738674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748a444-0039-4ce0-ade3-da67c9229853 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:08.739211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748a444-0039-4ce0-ade3-da67c9229853 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:14:08.739512 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748a444-0039-4ce0-ade3-da67c9229853 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:08.741100 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748a444-0039-4ce0-ade3-da67c9229853 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:08.753274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1094/2192] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:14:08.757815 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed776da-10f7-4f32-a95d-a00b676fc7c0 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:08.767401 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed776da-10f7-4f32-a95d-a00b676fc7c0 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:08.774186 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f3dc7-50cf-4a00-84ab-d2942fb7d6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:08.774824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f3dc7-50cf-4a00-84ab-d2942fb7d6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:08.775483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f3dc7-50cf-4a00-84ab-d2942fb7d6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:08.775483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f3dc7-50cf-4a00-84ab-d2942fb7d6e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:08.776132 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f3dc7-50cf-4a00-84ab-d2942fb7d6e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:08.778565 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f3dc7-50cf-4a00-84ab-d2942fb7d6e2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:08.813860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1099/2193] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:14:08 2022] POST /identity/v3/projects => generated 355 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:08.817982 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748a444-0039-4ce0-ade3-da67c9229853 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:08.829500 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748a444-0039-4ce0-ade3-da67c9229853 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:08.832277 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f3dc7-50cf-4a00-84ab-d2942fb7d6e2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:08.834469 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-aa5f3dc7-50cf-4a00-84ab-d2942fb7d6e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:08.845363 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:08.879618 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:08.912905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f3dc7-50cf-4a00-84ab-d2942fb7d6e2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:08.915208 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d3bb4-2997-4f80-8247-07a7424807ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:08.915958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d3bb4-2997-4f80-8247-07a7424807ee admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:08.916307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d3bb4-2997-4f80-8247-07a7424807ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:08.917524 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d3bb4-2997-4f80-8247-07a7424807ee admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:08.918182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d3bb4-2997-4f80-8247-07a7424807ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:08.919276 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5176af1-e3ce-426b-b31a-72818b73632b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:08.919911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176af1-e3ce-426b-b31a-72818b73632b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:08.920190 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176af1-e3ce-426b-b31a-72818b73632b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:08.920287 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d3bb4-2997-4f80-8247-07a7424807ee admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:08.920463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5176af1-e3ce-426b-b31a-72818b73632b admin admin] PATH_INFO: `/v3/projects/891e4951e79c40cc89e640d4aa1c3dd8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:08.921610 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5176af1-e3ce-426b-b31a-72818b73632b admin admin] RBAC: Authorizing `identity:delete_project(project_id=891e4951e79c40cc89e640d4aa1c3dd8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:08.925112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5176af1-e3ce-426b-b31a-72818b73632b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:08.946619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1095/2194] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:14:08 2022] POST /identity/v3/users => generated 278 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:08.953025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d3bb4-2997-4f80-8247-07a7424807ee admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:08.966259 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d3bb4-2997-4f80-8247-07a7424807ee admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:08.981435 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5176af1-e3ce-426b-b31a-72818b73632b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '891e4951e79c40cc89e640d4aa1c3dd8', 'request_id': 'req-f5176af1-e3ce-426b-b31a-72818b73632b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.990111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5176af1-e3ce-426b-b31a-72818b73632b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '891e4951e79c40cc89e640d4aa1c3dd8', 'request_id': 'req-f5176af1-e3ce-426b-b31a-72818b73632b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:08.997411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5176af1-e3ce-426b-b31a-72818b73632b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '891e4951e79c40cc89e640d4aa1c3dd8', 'request_id': 'req-f5176af1-e3ce-426b-b31a-72818b73632b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:09.014156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5176af1-e3ce-426b-b31a-72818b73632b admin admin] The token cache is being invalidate because project 891e4951e79c40cc89e640d4aa1c3dd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:14:09.014398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5176af1-e3ce-426b-b31a-72818b73632b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:09.015039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f5176af1-e3ce-426b-b31a-72818b73632b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:09.017036 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1100/2195] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:14:08 2022] DELETE /identity/v3/projects/891e4951e79c40cc89e640d4aa1c3dd8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:09.021630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e3cef-7c3b-4184-be42-457bc9b5d07c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.021915 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e3cef-7c3b-4184-be42-457bc9b5d07c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.022202 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722e3cef-7c3b-4184-be42-457bc9b5d07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.036752 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e224bd5-2ded-4af8-81b7-377e9506a204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.037510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e224bd5-2ded-4af8-81b7-377e9506a204 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.037933 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e224bd5-2ded-4af8-81b7-377e9506a204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.038310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e224bd5-2ded-4af8-81b7-377e9506a204 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.039191 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e224bd5-2ded-4af8-81b7-377e9506a204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.040797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e224bd5-2ded-4af8-81b7-377e9506a204 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:09.044548 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-722e3cef-7c3b-4184-be42-457bc9b5d07c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:09.045968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-722e3cef-7c3b-4184-be42-457bc9b5d07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:09.054773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1101/2196] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:14:09.062727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806d2ce-9d88-40f1-b392-a9cad063d1b2 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.063088 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806d2ce-9d88-40f1-b392-a9cad063d1b2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.063285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806d2ce-9d88-40f1-b392-a9cad063d1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.064113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1096/2197] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:14:09.065133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e224bd5-2ded-4af8-81b7-377e9506a204 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:09.066993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6e224bd5-2ded-4af8-81b7-377e9506a204 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:09.079947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e224bd5-2ded-4af8-81b7-377e9506a204 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:09.087491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2806d2ce-9d88-40f1-b392-a9cad063d1b2 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:09.087753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0303e6-b9e3-48ac-a863-ee9233032518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.088361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0303e6-b9e3-48ac-a863-ee9233032518 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.088634 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0303e6-b9e3-48ac-a863-ee9233032518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.088754 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2806d2ce-9d88-40f1-b392-a9cad063d1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:09.088952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0303e6-b9e3-48ac-a863-ee9233032518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.090189 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0303e6-b9e3-48ac-a863-ee9233032518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.091769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0303e6-b9e3-48ac-a863-ee9233032518 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:09.098041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1102/2198] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:09.102397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806d2ce-9d88-40f1-b392-a9cad063d1b2 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:09.105354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1097/2199] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:14:09.106540 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0303e6-b9e3-48ac-a863-ee9233032518 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:09.111910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806d2ce-9d88-40f1-b392-a9cad063d1b2 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:09.117351 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363b1be-1b5c-4a85-a1de-1b28e87c29aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.117954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363b1be-1b5c-4a85-a1de-1b28e87c29aa admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.118234 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363b1be-1b5c-4a85-a1de-1b28e87c29aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.118507 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363b1be-1b5c-4a85-a1de-1b28e87c29aa admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.119147 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363b1be-1b5c-4a85-a1de-1b28e87c29aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:14:09.119309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0303e6-b9e3-48ac-a863-ee9233032518 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:09.119460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363b1be-1b5c-4a85-a1de-1b28e87c29aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.121617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363b1be-1b5c-4a85-a1de-1b28e87c29aa admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:09.126051 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323db6f-5ef9-4cf3-a20b-8513ed00aa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.127092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323db6f-5ef9-4cf3-a20b-8513ed00aa24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.127461 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323db6f-5ef9-4cf3-a20b-8513ed00aa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.127888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323db6f-5ef9-4cf3-a20b-8513ed00aa24 admin admin] PATH_INFO: `/v3/projects/80f8d7ca20ed4c8bbb167197ddbbfdaf/users/5407539cb9224a5284765103c7c999ca/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.136572 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1103/2200] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:14:09.141097 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363b1be-1b5c-4a85-a1de-1b28e87c29aa admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:09.147119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323db6f-5ef9-4cf3-a20b-8513ed00aa24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80f8d7ca20ed4c8bbb167197ddbbfdaf, user_id=5407539cb9224a5284765103c7c999ca, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.149875 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323db6f-5ef9-4cf3-a20b-8513ed00aa24 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:09.152721 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363b1be-1b5c-4a85-a1de-1b28e87c29aa admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:09.158587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81abde2-de2d-4983-ac7e-3205974f774e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.159193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81abde2-de2d-4983-ac7e-3205974f774e admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.159513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81abde2-de2d-4983-ac7e-3205974f774e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.159824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81abde2-de2d-4983-ac7e-3205974f774e admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.160618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81abde2-de2d-4983-ac7e-3205974f774e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.162708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81abde2-de2d-4983-ac7e-3205974f774e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:09.168998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1098/2201] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:09 2022] PUT /identity/v3/projects/80f8d7ca20ed4c8bbb167197ddbbfdaf/users/5407539cb9224a5284765103c7c999ca/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:14:09.178299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dc967-0e8c-41e9-bb39-0b9608c66fad None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.178815 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dc967-0e8c-41e9-bb39-0b9608c66fad None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.179367 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44dc967-0e8c-41e9-bb39-0b9608c66fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.196917 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1104/2202] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:14:09 2022] POST /identity/v3/projects => generated 363 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:09.198976 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81abde2-de2d-4983-ac7e-3205974f774e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:09.220136 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a44dc967-0e8c-41e9-bb39-0b9608c66fad None None] MFA Rules not processed for user `5407539cb9224a5284765103c7c999ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:09.221560 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a44dc967-0e8c-41e9-bb39-0b9608c66fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:09.249895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:09.266339 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81abde2-de2d-4983-ac7e-3205974f774e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:09.272021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0f450-15ab-4998-906a-5d723298dcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.272839 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0f450-15ab-4998-906a-5d723298dcc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.273122 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0f450-15ab-4998-906a-5d723298dcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.273427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0f450-15ab-4998-906a-5d723298dcc0 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.274049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0f450-15ab-4998-906a-5d723298dcc0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.275989 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0f450-15ab-4998-906a-5d723298dcc0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:09.287977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1099/2203] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:09.292613 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44dc967-0e8c-41e9-bb39-0b9608c66fad None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:09.300905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1105/2204] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:14:09 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:09.302124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0f450-15ab-4998-906a-5d723298dcc0 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:09.303675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44dc967-0e8c-41e9-bb39-0b9608c66fad None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:09.309682 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4af955-7603-4849-9d95-619ffbce6a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.310283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4af955-7603-4849-9d95-619ffbce6a42 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.310560 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4af955-7603-4849-9d95-619ffbce6a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.310834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4af955-7603-4849-9d95-619ffbce6a42 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.311451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4af955-7603-4849-9d95-619ffbce6a42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.312448 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0f450-15ab-4998-906a-5d723298dcc0 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:09.313372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4af955-7603-4849-9d95-619ffbce6a42 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:09.317258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cb556-ff5f-4437-8a0a-442385b9356e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.317839 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cb556-ff5f-4437-8a0a-442385b9356e admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.318113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cb556-ff5f-4437-8a0a-442385b9356e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.318382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cb556-ff5f-4437-8a0a-442385b9356e admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.319041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cb556-ff5f-4437-8a0a-442385b9356e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.320591 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cb556-ff5f-4437-8a0a-442385b9356e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:09.325265 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:09.333236 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1106/2205] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:14:09.335640 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cb556-ff5f-4437-8a0a-442385b9356e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:09.345498 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1100/2206] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:14:09 2022] POST /identity/v3/projects => generated 357 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:14:09.345966 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cb556-ff5f-4437-8a0a-442385b9356e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:09.346371 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4af955-7603-4849-9d95-619ffbce6a42 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:09.403370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d78b7-8062-474b-95c9-56877b868b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.403975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d78b7-8062-474b-95c9-56877b868b4f admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.404260 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d78b7-8062-474b-95c9-56877b868b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.404535 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20d78b7-8062-474b-95c9-56877b868b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.405169 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d78b7-8062-474b-95c9-56877b868b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.406781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20d78b7-8062-474b-95c9-56877b868b4f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:09.411552 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4af955-7603-4849-9d95-619ffbce6a42 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:09.416112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983758d3-7644-4041-ba27-0dd913e650f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.416402 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1107/2207] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:14:09.416706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983758d3-7644-4041-ba27-0dd913e650f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.417148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983758d3-7644-4041-ba27-0dd913e650f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.417443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983758d3-7644-4041-ba27-0dd913e650f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.418059 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983758d3-7644-4041-ba27-0dd913e650f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.419984 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983758d3-7644-4041-ba27-0dd913e650f2 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:09.420549 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d78b7-8062-474b-95c9-56877b868b4f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:09.430164 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20d78b7-8062-474b-95c9-56877b868b4f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:09.437662 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9263b67c-3137-46e1-ad7f-11ea32696acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.438306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263b67c-3137-46e1-ad7f-11ea32696acc admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.438584 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263b67c-3137-46e1-ad7f-11ea32696acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.438853 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9263b67c-3137-46e1-ad7f-11ea32696acc admin admin] PATH_INFO: `/v3/projects/5a5946d4b3564b439464a90ad9334bb5/users/12f8acad04b04f499d40606a243fec5b/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.442627 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1101/2208] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:14:09 2022] POST /identity/v3/users => generated 285 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:14:09.446481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983758d3-7644-4041-ba27-0dd913e650f2 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:09.455345 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263b67c-3137-46e1-ad7f-11ea32696acc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a5946d4b3564b439464a90ad9334bb5, user_id=12f8acad04b04f499d40606a243fec5b, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.456744 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983758d3-7644-4041-ba27-0dd913e650f2 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:09.458074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9263b67c-3137-46e1-ad7f-11ea32696acc admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:09.461430 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331e83f-8b6d-4c07-a11e-5dc2621fa345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.462019 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331e83f-8b6d-4c07-a11e-5dc2621fa345 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.462334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331e83f-8b6d-4c07-a11e-5dc2621fa345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.463047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c331e83f-8b6d-4c07-a11e-5dc2621fa345 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.463955 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331e83f-8b6d-4c07-a11e-5dc2621fa345 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.465994 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c331e83f-8b6d-4c07-a11e-5dc2621fa345 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:09.471158 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1108/2209] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:09 2022] PUT /identity/v3/projects/5a5946d4b3564b439464a90ad9334bb5/users/12f8acad04b04f499d40606a243fec5b/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:14:09.481041 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1102/2210] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:14:09.481270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab623db2-09d4-4051-b596-3da25dba477f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.481567 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab623db2-09d4-4051-b596-3da25dba477f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.481845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab623db2-09d4-4051-b596-3da25dba477f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.482290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331e83f-8b6d-4c07-a11e-5dc2621fa345 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:09.492181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c331e83f-8b6d-4c07-a11e-5dc2621fa345 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:09.512092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ab623db2-09d4-4051-b596-3da25dba477f None None] MFA Rules not processed for user `12f8acad04b04f499d40606a243fec5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:09.513582 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ab623db2-09d4-4051-b596-3da25dba477f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:09.549408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03859b20-e9a3-4d3f-aa25-53084b36831c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.549984 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859b20-e9a3-4d3f-aa25-53084b36831c admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.550266 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859b20-e9a3-4d3f-aa25-53084b36831c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.550541 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03859b20-e9a3-4d3f-aa25-53084b36831c admin admin] PATH_INFO: `/v3/projects/cfe1d38f14674e6da2f685b10ee0361b/users/33d5ed55606241258504bb66ac70ae8c/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.568265 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03859b20-e9a3-4d3f-aa25-53084b36831c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe1d38f14674e6da2f685b10ee0361b, user_id=33d5ed55606241258504bb66ac70ae8c, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.571339 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03859b20-e9a3-4d3f-aa25-53084b36831c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:09.583311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1109/2211] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:09.590299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1103/2212] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:09 2022] PUT /identity/v3/projects/cfe1d38f14674e6da2f685b10ee0361b/users/33d5ed55606241258504bb66ac70ae8c/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:09.594313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab623db2-09d4-4051-b596-3da25dba477f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:09.595015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03859b20-e9a3-4d3f-aa25-53084b36831c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:09.596813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-03859b20-e9a3-4d3f-aa25-53084b36831c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:09.609547 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab623db2-09d4-4051-b596-3da25dba477f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:09.638532 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:09.664902 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03859b20-e9a3-4d3f-aa25-53084b36831c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:09.669618 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad3253-4b2a-4f2a-a163-f1550de1ed9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.670183 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a6f7c-03be-4c51-9dea-5d48563d2518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.670345 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad3253-4b2a-4f2a-a163-f1550de1ed9e service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.670663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad3253-4b2a-4f2a-a163-f1550de1ed9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.670782 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a6f7c-03be-4c51-9dea-5d48563d2518 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.670931 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad3253-4b2a-4f2a-a163-f1550de1ed9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.671053 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a6f7c-03be-4c51-9dea-5d48563d2518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.671332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293a6f7c-03be-4c51-9dea-5d48563d2518 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.671968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a6f7c-03be-4c51-9dea-5d48563d2518 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.673591 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293a6f7c-03be-4c51-9dea-5d48563d2518 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:09.676289 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:09.684666 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1110/2213] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:09 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:14:09.690203 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a6f7c-03be-4c51-9dea-5d48563d2518 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:09.704462 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293a6f7c-03be-4c51-9dea-5d48563d2518 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:09.709163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590f43e-8539-4608-b83f-d7c66ca0ebaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.709847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f43e-8539-4608-b83f-d7c66ca0ebaa admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.710124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f43e-8539-4608-b83f-d7c66ca0ebaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.710395 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590f43e-8539-4608-b83f-d7c66ca0ebaa admin admin] PATH_INFO: `/v3/projects/cfe1d38f14674e6da2f685b10ee0361b/users/33d5ed55606241258504bb66ac70ae8c/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.712449 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590f43e-8539-4608-b83f-d7c66ca0ebaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe1d38f14674e6da2f685b10ee0361b, user_id=33d5ed55606241258504bb66ac70ae8c, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.715167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4590f43e-8539-4608-b83f-d7c66ca0ebaa admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:09.725453 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1111/2214] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:09 2022] PUT /identity/v3/projects/cfe1d38f14674e6da2f685b10ee0361b/users/33d5ed55606241258504bb66ac70ae8c/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:09.734954 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad3253-4b2a-4f2a-a163-f1550de1ed9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.736342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ad3253-4b2a-4f2a-a163-f1550de1ed9e service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:09.736547 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d2123-c00d-4f47-a35b-4b7f1657e3a7 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.737003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d2123-c00d-4f47-a35b-4b7f1657e3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.737302 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5d2123-c00d-4f47-a35b-4b7f1657e3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.746440 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1104/2215] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:09 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:14:09.769207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0e5d2123-c00d-4f47-a35b-4b7f1657e3a7 None None] MFA Rules not processed for user `33d5ed55606241258504bb66ac70ae8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:09.770450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0e5d2123-c00d-4f47-a35b-4b7f1657e3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:09.844617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1112/2216] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:09 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:09.911589 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad3253-4b2a-4f2a-a163-f1550de1ed9e service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:09.913317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e5ad3253-4b2a-4f2a-a163-f1550de1ed9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:09.925877 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad3253-4b2a-4f2a-a163-f1550de1ed9e service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:09.977453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43615d43-352d-4b23-b533-9f77d18967ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:09.978052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43615d43-352d-4b23-b533-9f77d18967ee service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:09.978329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43615d43-352d-4b23-b533-9f77d18967ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:09.978592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43615d43-352d-4b23-b533-9f77d18967ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:09.990157 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43615d43-352d-4b23-b533-9f77d18967ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:09.991648 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43615d43-352d-4b23-b533-9f77d18967ee service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:10.014304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1105/2217] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:09 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:14:10.108864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d2123-c00d-4f47-a35b-4b7f1657e3a7 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:10.120848 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5d2123-c00d-4f47-a35b-4b7f1657e3a7 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:10.124753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ceb103-ad1a-4c76-999b-b6fb374a305a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:10.125397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ceb103-ad1a-4c76-999b-b6fb374a305a service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:10.125894 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ceb103-ad1a-4c76-999b-b6fb374a305a service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:10.126350 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ceb103-ad1a-4c76-999b-b6fb374a305a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:10.189171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ceb103-ad1a-4c76-999b-b6fb374a305a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:10.190750 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ceb103-ad1a-4c76-999b-b6fb374a305a service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:10.203056 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1113/2218] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:10 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:14:18.098899 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43615d43-352d-4b23-b533-9f77d18967ee service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:18.100798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-43615d43-352d-4b23-b533-9f77d18967ee service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:18.177661 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43615d43-352d-4b23-b533-9f77d18967ee service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:18.182743 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8c92e-7867-4c9b-9245-ab41ab896835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:18.183885 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8c92e-7867-4c9b-9245-ab41ab896835 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:18.184155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8c92e-7867-4c9b-9245-ab41ab896835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:18.184419 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd8c92e-7867-4c9b-9245-ab41ab896835 admin admin] PATH_INFO: `/v3/users/49c9f15edb16431dbd39494df0d47dd5` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:18.185612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8c92e-7867-4c9b-9245-ab41ab896835 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49c9f15edb16431dbd39494df0d47dd5)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:18.187551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd8c92e-7867-4c9b-9245-ab41ab896835 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:18.245985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3cd8c92e-7867-4c9b-9245-ab41ab896835 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49c9f15edb16431dbd39494df0d47dd5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:18.251346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:18.255434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3cd8c92e-7867-4c9b-9245-ab41ab896835 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49c9f15edb16431dbd39494df0d47dd5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:18.267743 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3cd8c92e-7867-4c9b-9245-ab41ab896835 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49c9f15edb16431dbd39494df0d47dd5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:18.268410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3cd8c92e-7867-4c9b-9245-ab41ab896835 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49c9f15edb16431dbd39494df0d47dd5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:18.269012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-3cd8c92e-7867-4c9b-9245-ab41ab896835 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49c9f15edb16431dbd39494df0d47dd5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:18.283527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1106/2219] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:14:18 2022] DELETE /identity/v3/users/49c9f15edb16431dbd39494df0d47dd5 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:18.288097 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7799e-f784-46ee-8a81-56142bc73e76 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:18.288550 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7799e-f784-46ee-8a81-56142bc73e76 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:18.288685 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d7799e-f784-46ee-8a81-56142bc73e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:18.308320 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-27d7799e-f784-46ee-8a81-56142bc73e76 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:18.309541 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-27d7799e-f784-46ee-8a81-56142bc73e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:18.337941 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:18.368633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1114/2220] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:18.378769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8c92e-7867-4c9b-9245-ab41ab896835 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:18.380256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3cd8c92e-7867-4c9b-9245-ab41ab896835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:18.450286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd8c92e-7867-4c9b-9245-ab41ab896835 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:18.454288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274c4f11-8dda-4d7d-b840-bd8e440fe39d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:18.454894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c4f11-8dda-4d7d-b840-bd8e440fe39d service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:18.455172 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c4f11-8dda-4d7d-b840-bd8e440fe39d service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:18.455444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274c4f11-8dda-4d7d-b840-bd8e440fe39d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:18.465903 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c4f11-8dda-4d7d-b840-bd8e440fe39d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:18.467486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274c4f11-8dda-4d7d-b840-bd8e440fe39d service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:18.477955 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1107/2221] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:14:18.805312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7799e-f784-46ee-8a81-56142bc73e76 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:18.806816 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-27d7799e-f784-46ee-8a81-56142bc73e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:18.825180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d7799e-f784-46ee-8a81-56142bc73e76 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:18.837269 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e18673-d7be-4aea-8d69-c6c85f486f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:18.838070 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e18673-d7be-4aea-8d69-c6c85f486f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:18.838268 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e18673-d7be-4aea-8d69-c6c85f486f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:18.838491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e18673-d7be-4aea-8d69-c6c85f486f8c admin admin] PATH_INFO: `/v3/projects/db718e30a3d54deaab20dd3a17279fe9` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:18.840253 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e18673-d7be-4aea-8d69-c6c85f486f8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=db718e30a3d54deaab20dd3a17279fe9)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:18.846513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e18673-d7be-4aea-8d69-c6c85f486f8c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:18.921634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-67e18673-d7be-4aea-8d69-c6c85f486f8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db718e30a3d54deaab20dd3a17279fe9', 'request_id': 'req-67e18673-d7be-4aea-8d69-c6c85f486f8c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:18.929126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-67e18673-d7be-4aea-8d69-c6c85f486f8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db718e30a3d54deaab20dd3a17279fe9', 'request_id': 'req-67e18673-d7be-4aea-8d69-c6c85f486f8c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:18.936543 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-67e18673-d7be-4aea-8d69-c6c85f486f8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db718e30a3d54deaab20dd3a17279fe9', 'request_id': 'req-67e18673-d7be-4aea-8d69-c6c85f486f8c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:18.955993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-67e18673-d7be-4aea-8d69-c6c85f486f8c admin admin] The token cache is being invalidate because project db718e30a3d54deaab20dd3a17279fe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:14:18.956341 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-67e18673-d7be-4aea-8d69-c6c85f486f8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:18.957043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-67e18673-d7be-4aea-8d69-c6c85f486f8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:18.959174 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1115/2222] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:14:18 2022] DELETE /identity/v3/projects/db718e30a3d54deaab20dd3a17279fe9 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:18.964645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b6650-6d20-4238-b71f-4346eb89d921 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:18.965079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b6650-6d20-4238-b71f-4346eb89d921 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:18.965465 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b6650-6d20-4238-b71f-4346eb89d921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:18.990100 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-554b6650-6d20-4238-b71f-4346eb89d921 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:18.991472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-554b6650-6d20-4238-b71f-4346eb89d921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:19.097927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1108/2223] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:14:19.106358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d7ff0-0115-4ba2-83f3-d97210169ce5 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:19.106788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d7ff0-0115-4ba2-83f3-d97210169ce5 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:19.107469 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2d7ff0-0115-4ba2-83f3-d97210169ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:19.131990 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0f2d7ff0-0115-4ba2-83f3-d97210169ce5 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:19.133550 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0f2d7ff0-0115-4ba2-83f3-d97210169ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:19.144587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1116/2224] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:19.149315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b6650-6d20-4238-b71f-4346eb89d921 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:19.163150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b6650-6d20-4238-b71f-4346eb89d921 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:19.168985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc705e-454a-4a81-b41a-672d623b680e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:19.170060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc705e-454a-4a81-b41a-672d623b680e admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:19.170438 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc705e-454a-4a81-b41a-672d623b680e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:19.170918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cc705e-454a-4a81-b41a-672d623b680e admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:19.171999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc705e-454a-4a81-b41a-672d623b680e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:14:19.172477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc705e-454a-4a81-b41a-672d623b680e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:19.174573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cc705e-454a-4a81-b41a-672d623b680e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:19.196427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1109/2225] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:14:19.197163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d7ff0-0115-4ba2-83f3-d97210169ce5 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:19.218726 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2d7ff0-0115-4ba2-83f3-d97210169ce5 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:19.229000 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4961e3b-89b5-42d9-ae6c-4fafc4f6290e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:19.230029 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4961e3b-89b5-42d9-ae6c-4fafc4f6290e admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:19.230314 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4961e3b-89b5-42d9-ae6c-4fafc4f6290e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:19.230601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4961e3b-89b5-42d9-ae6c-4fafc4f6290e admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:19.231262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4961e3b-89b5-42d9-ae6c-4fafc4f6290e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:19.235644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4961e3b-89b5-42d9-ae6c-4fafc4f6290e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:19.285258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1117/2226] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:14:19 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:19.287098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc705e-454a-4a81-b41a-672d623b680e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:19.297510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cc705e-454a-4a81-b41a-672d623b680e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:19.355185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384f0f6-43d5-4061-97c7-c4ab157eb2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:19.355779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384f0f6-43d5-4061-97c7-c4ab157eb2f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:19.356054 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384f0f6-43d5-4061-97c7-c4ab157eb2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:19.356323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2384f0f6-43d5-4061-97c7-c4ab157eb2f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:19.357525 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384f0f6-43d5-4061-97c7-c4ab157eb2f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:19.359465 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2384f0f6-43d5-4061-97c7-c4ab157eb2f1 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:19.367397 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:19.383536 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1110/2227] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:14:19 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:19.388211 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4961e3b-89b5-42d9-ae6c-4fafc4f6290e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:19.399632 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4961e3b-89b5-42d9-ae6c-4fafc4f6290e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:19.404419 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cfaec-7811-468c-b0c7-b028bbfff282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:19.405020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cfaec-7811-468c-b0c7-b028bbfff282 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:19.405337 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cfaec-7811-468c-b0c7-b028bbfff282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:19.405604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756cfaec-7811-468c-b0c7-b028bbfff282 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:19.406247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756cfaec-7811-468c-b0c7-b028bbfff282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:19.407835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756cfaec-7811-468c-b0c7-b028bbfff282 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:19.419942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1118/2228] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:14:19.424008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384f0f6-43d5-4061-97c7-c4ab157eb2f1 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:19.434706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2384f0f6-43d5-4061-97c7-c4ab157eb2f1 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:19.439321 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c15420f-b1b3-4999-b43b-3a2175fec6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:19.439908 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15420f-b1b3-4999-b43b-3a2175fec6cf admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:19.440183 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15420f-b1b3-4999-b43b-3a2175fec6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:19.440454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c15420f-b1b3-4999-b43b-3a2175fec6cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:19.441320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15420f-b1b3-4999-b43b-3a2175fec6cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:19.442773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c15420f-b1b3-4999-b43b-3a2175fec6cf admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:19.454332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1111/2229] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:14:19.458824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cfaec-7811-468c-b0c7-b028bbfff282 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:19.471834 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756cfaec-7811-468c-b0c7-b028bbfff282 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:19.477390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c244571-519d-4fb2-803e-c6754ed09abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:19.488317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c244571-519d-4fb2-803e-c6754ed09abf admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:19.488622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c244571-519d-4fb2-803e-c6754ed09abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:19.488900 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c244571-519d-4fb2-803e-c6754ed09abf admin admin] PATH_INFO: `/v3/projects/fe3b687e68734a8888291f0c3527acbc/users/26cd75e2c118412a88bb354e18863d34/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:19.509322 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c244571-519d-4fb2-803e-c6754ed09abf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe3b687e68734a8888291f0c3527acbc, user_id=26cd75e2c118412a88bb354e18863d34, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:19.513170 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c244571-519d-4fb2-803e-c6754ed09abf admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:19.532226 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1119/2230] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:19 2022] PUT /identity/v3/projects/fe3b687e68734a8888291f0c3527acbc/users/26cd75e2c118412a88bb354e18863d34/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:19.537427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7fec3-a509-4cb3-852a-0e58ed9e2471 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:19.537705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7fec3-a509-4cb3-852a-0e58ed9e2471 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:19.537999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7fec3-a509-4cb3-852a-0e58ed9e2471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:19.557103 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-5ad7fec3-a509-4cb3-852a-0e58ed9e2471 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:19.558813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5ad7fec3-a509-4cb3-852a-0e58ed9e2471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:19.620296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1112/2231] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:19.630250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c244571-519d-4fb2-803e-c6754ed09abf admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:19.631949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2c244571-519d-4fb2-803e-c6754ed09abf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:19.688644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c244571-519d-4fb2-803e-c6754ed09abf admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:19.692190 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf9937-4e8c-41f8-b898-6bb135443f7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:19.693067 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf9937-4e8c-41f8-b898-6bb135443f7a service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:19.693067 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf9937-4e8c-41f8-b898-6bb135443f7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:19.693353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bf9937-4e8c-41f8-b898-6bb135443f7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:19.705198 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf9937-4e8c-41f8-b898-6bb135443f7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:19.706630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bf9937-4e8c-41f8-b898-6bb135443f7a service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:19.710794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:19.718732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1120/2232] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:14:24.607709 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459155bc-8423-438d-ae62-30b56138b448 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:24.609136 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459155bc-8423-438d-ae62-30b56138b448 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:24.609666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459155bc-8423-438d-ae62-30b56138b448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:24.651013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-459155bc-8423-438d-ae62-30b56138b448 None None] MFA Rules not processed for user `26cd75e2c118412a88bb354e18863d34`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:24.652856 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-459155bc-8423-438d-ae62-30b56138b448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:24.746202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1113/2233] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:24.750743 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf9937-4e8c-41f8-b898-6bb135443f7a service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:24.761335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bf9937-4e8c-41f8-b898-6bb135443f7a service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:24.767566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bc350-48ea-4320-874e-4905c7c4ce61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:24.768181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bc350-48ea-4320-874e-4905c7c4ce61 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:24.768455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bc350-48ea-4320-874e-4905c7c4ce61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:24.768725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5bc350-48ea-4320-874e-4905c7c4ce61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:24.769400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bc350-48ea-4320-874e-4905c7c4ce61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:24.771315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5bc350-48ea-4320-874e-4905c7c4ce61 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:24.805830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1121/2234] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:14:24 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:24.807906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459155bc-8423-438d-ae62-30b56138b448 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:24.819636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459155bc-8423-438d-ae62-30b56138b448 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:24.884089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7158fe-509a-48db-be59-5f54870e5371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:24.884709 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7158fe-509a-48db-be59-5f54870e5371 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:24.884984 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7158fe-509a-48db-be59-5f54870e5371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:24.885290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7158fe-509a-48db-be59-5f54870e5371 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:24.886072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7158fe-509a-48db-be59-5f54870e5371 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:24.888004 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7158fe-509a-48db-be59-5f54870e5371 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:24.906077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:24.917770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1114/2235] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:14:24 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:24.918113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bc350-48ea-4320-874e-4905c7c4ce61 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:24.929919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5bc350-48ea-4320-874e-4905c7c4ce61 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:24.936042 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d491507-fcdb-450f-83af-d0dd4e06f292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:24.937182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d491507-fcdb-450f-83af-d0dd4e06f292 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:24.937690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d491507-fcdb-450f-83af-d0dd4e06f292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:24.938178 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d491507-fcdb-450f-83af-d0dd4e06f292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:24.939062 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d491507-fcdb-450f-83af-d0dd4e06f292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:24.940738 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d491507-fcdb-450f-83af-d0dd4e06f292 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:24.953339 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1122/2236] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:14:24.957780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7158fe-509a-48db-be59-5f54870e5371 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:24.970095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7158fe-509a-48db-be59-5f54870e5371 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:24.974966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc83fc-921b-445e-8d8e-0b73ec106ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:24.975786 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc83fc-921b-445e-8d8e-0b73ec106ac6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:24.975956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc83fc-921b-445e-8d8e-0b73ec106ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:24.976234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc83fc-921b-445e-8d8e-0b73ec106ac6 admin admin] PATH_INFO: `/v3/projects/b19d4a8c14834bc79ebedc8496a1982d/users/eb940cd2e00c455c947e0bc0ad4c4d97/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:24.998588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc83fc-921b-445e-8d8e-0b73ec106ac6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19d4a8c14834bc79ebedc8496a1982d, user_id=eb940cd2e00c455c947e0bc0ad4c4d97, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:25.001404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc83fc-921b-445e-8d8e-0b73ec106ac6 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:25.015754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1115/2237] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:24 2022] PUT /identity/v3/projects/b19d4a8c14834bc79ebedc8496a1982d/users/eb940cd2e00c455c947e0bc0ad4c4d97/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:25.019960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d491507-fcdb-450f-83af-d0dd4e06f292 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:25.030717 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d491507-fcdb-450f-83af-d0dd4e06f292 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:25.093667 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d563020-e610-480b-bf6c-db03e5e3f67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:25.094201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d563020-e610-480b-bf6c-db03e5e3f67c admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:25.094479 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d563020-e610-480b-bf6c-db03e5e3f67c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:25.094749 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d563020-e610-480b-bf6c-db03e5e3f67c admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:25.095582 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d563020-e610-480b-bf6c-db03e5e3f67c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:25.097158 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d563020-e610-480b-bf6c-db03e5e3f67c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:25.108530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1123/2238] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:14:25.112983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc83fc-921b-445e-8d8e-0b73ec106ac6 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:25.124257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc83fc-921b-445e-8d8e-0b73ec106ac6 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:25.128979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0b8b3-e061-4185-a76b-0bb04209389c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:25.129660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0b8b3-e061-4185-a76b-0bb04209389c admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:25.129935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0b8b3-e061-4185-a76b-0bb04209389c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:25.130202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e0b8b3-e061-4185-a76b-0bb04209389c admin admin] PATH_INFO: `/v3/projects/b19d4a8c14834bc79ebedc8496a1982d/users/eb940cd2e00c455c947e0bc0ad4c4d97/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:25.132083 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0b8b3-e061-4185-a76b-0bb04209389c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19d4a8c14834bc79ebedc8496a1982d, user_id=eb940cd2e00c455c947e0bc0ad4c4d97, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:25.134716 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e0b8b3-e061-4185-a76b-0bb04209389c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:25.145752 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1116/2239] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:25 2022] PUT /identity/v3/projects/b19d4a8c14834bc79ebedc8496a1982d/users/eb940cd2e00c455c947e0bc0ad4c4d97/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:31.074826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f5a4a-3689-438d-91cc-aca831635d99 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:31.075384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f5a4a-3689-438d-91cc-aca831635d99 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:31.075384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f5a4a-3689-438d-91cc-aca831635d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:31.123402 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-603f5a4a-3689-438d-91cc-aca831635d99 None None] MFA Rules not processed for user `eb940cd2e00c455c947e0bc0ad4c4d97`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:31.125124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-603f5a4a-3689-438d-91cc-aca831635d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:31.220706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1124/2240] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:31.234097 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0b8b3-e061-4185-a76b-0bb04209389c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:31.235966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-82e0b8b3-e061-4185-a76b-0bb04209389c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:31.321450 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e0b8b3-e061-4185-a76b-0bb04209389c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:31.327590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a618e04-fa63-4748-9357-4f403550313b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:31.328261 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a618e04-fa63-4748-9357-4f403550313b service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:31.328537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a618e04-fa63-4748-9357-4f403550313b service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:31.328819 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a618e04-fa63-4748-9357-4f403550313b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:31.383841 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a618e04-fa63-4748-9357-4f403550313b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:31.386153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a618e04-fa63-4748-9357-4f403550313b service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:31.394955 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1117/2241] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:14:32.469980 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000ef6d-267d-4a14-85b5-25466dd7436d None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:32.470433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000ef6d-267d-4a14-85b5-25466dd7436d None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:32.470551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b000ef6d-267d-4a14-85b5-25466dd7436d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:32.493613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b000ef6d-267d-4a14-85b5-25466dd7436d None None] MFA Rules not processed for user `26cd75e2c118412a88bb354e18863d34`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:32.495476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b000ef6d-267d-4a14-85b5-25466dd7436d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:32.505232 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1125/2242] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:37.784566 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a618e04-fa63-4748-9357-4f403550313b service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:37.787203 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8a618e04-fa63-4748-9357-4f403550313b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:37.891664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:37.911792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a618e04-fa63-4748-9357-4f403550313b service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:37.918919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a68a8-420d-475f-8be2-6f591462aa87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:37.919661 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a68a8-420d-475f-8be2-6f591462aa87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:37.919943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a68a8-420d-475f-8be2-6f591462aa87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:37.920281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a68a8-420d-475f-8be2-6f591462aa87 admin admin] PATH_INFO: `/v3/users/79ee3ada54674744b9a3126930d97caa` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:37.921673 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a68a8-420d-475f-8be2-6f591462aa87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79ee3ada54674744b9a3126930d97caa)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:37.923816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0a68a8-420d-475f-8be2-6f591462aa87 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:37.985227 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ad0a68a8-420d-475f-8be2-6f591462aa87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79ee3ada54674744b9a3126930d97caa'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:37.992587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ad0a68a8-420d-475f-8be2-6f591462aa87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79ee3ada54674744b9a3126930d97caa'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:38.005612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ad0a68a8-420d-475f-8be2-6f591462aa87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79ee3ada54674744b9a3126930d97caa'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:38.006329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ad0a68a8-420d-475f-8be2-6f591462aa87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79ee3ada54674744b9a3126930d97caa'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:38.006961 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ad0a68a8-420d-475f-8be2-6f591462aa87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79ee3ada54674744b9a3126930d97caa'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:38.019770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1118/2243] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:14:37 2022] DELETE /identity/v3/users/79ee3ada54674744b9a3126930d97caa => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:38.026242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f7bb9-3dc0-4cb4-9f18-e6409418c1a0 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:38.026793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f7bb9-3dc0-4cb4-9f18-e6409418c1a0 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:38.026793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63f7bb9-3dc0-4cb4-9f18-e6409418c1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:38.049388 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c63f7bb9-3dc0-4cb4-9f18-e6409418c1a0 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:38.050732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c63f7bb9-3dc0-4cb4-9f18-e6409418c1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:38.085793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:38.115251 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1126/2244] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:38.125562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a68a8-420d-475f-8be2-6f591462aa87 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:38.127077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ad0a68a8-420d-475f-8be2-6f591462aa87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:38.203046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a68a8-420d-475f-8be2-6f591462aa87 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:38.206996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30dcc7-85b8-4fc1-bbdf-bb3eebcf18a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:38.207598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30dcc7-85b8-4fc1-bbdf-bb3eebcf18a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:38.207874 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30dcc7-85b8-4fc1-bbdf-bb3eebcf18a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:38.208149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30dcc7-85b8-4fc1-bbdf-bb3eebcf18a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:38.217426 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30dcc7-85b8-4fc1-bbdf-bb3eebcf18a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:38.218857 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30dcc7-85b8-4fc1-bbdf-bb3eebcf18a6 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:38.228239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1119/2245] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:14:38.611319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f7bb9-3dc0-4cb4-9f18-e6409418c1a0 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:38.612770 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c63f7bb9-3dc0-4cb4-9f18-e6409418c1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:38.628825 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63f7bb9-3dc0-4cb4-9f18-e6409418c1a0 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:38.636769 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fdb9f-8f4a-4986-8c29-f92185d23431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:38.637424 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fdb9f-8f4a-4986-8c29-f92185d23431 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:38.637709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fdb9f-8f4a-4986-8c29-f92185d23431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:38.637979 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fdb9f-8f4a-4986-8c29-f92185d23431 admin admin] PATH_INFO: `/v3/projects/3d581d500fdb44e4a3845dfcecc7d498` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:38.640026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fdb9f-8f4a-4986-8c29-f92185d23431 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d581d500fdb44e4a3845dfcecc7d498)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:38.642763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fdb9f-8f4a-4986-8c29-f92185d23431 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:38.706258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-097fdb9f-8f4a-4986-8c29-f92185d23431 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d581d500fdb44e4a3845dfcecc7d498', 'request_id': 'req-097fdb9f-8f4a-4986-8c29-f92185d23431'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:38.713416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-097fdb9f-8f4a-4986-8c29-f92185d23431 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d581d500fdb44e4a3845dfcecc7d498', 'request_id': 'req-097fdb9f-8f4a-4986-8c29-f92185d23431'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:38.719538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-097fdb9f-8f4a-4986-8c29-f92185d23431 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d581d500fdb44e4a3845dfcecc7d498', 'request_id': 'req-097fdb9f-8f4a-4986-8c29-f92185d23431'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:38.736033 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30dcc7-85b8-4fc1-bbdf-bb3eebcf18a6 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:38.737171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-097fdb9f-8f4a-4986-8c29-f92185d23431 admin admin] The token cache is being invalidate because project 3d581d500fdb44e4a3845dfcecc7d498 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:14:38.737507 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-097fdb9f-8f4a-4986-8c29-f92185d23431 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:38.738054 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3b30dcc7-85b8-4fc1-bbdf-bb3eebcf18a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:38.738256 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-097fdb9f-8f4a-4986-8c29-f92185d23431 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:38.739759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1127/2246] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:14:38 2022] DELETE /identity/v3/projects/3d581d500fdb44e4a3845dfcecc7d498 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:38.745368 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a94a52-a407-433a-940f-2d361bce6d66 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:38.745778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a94a52-a407-433a-940f-2d361bce6d66 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:38.746288 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a94a52-a407-433a-940f-2d361bce6d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:38.759452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30dcc7-85b8-4fc1-bbdf-bb3eebcf18a6 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:38.779566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c6a94a52-a407-433a-940f-2d361bce6d66 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:38.780805 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c6a94a52-a407-433a-940f-2d361bce6d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:38.827802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c880fa-a20a-4a8a-aa36-a0d8e1eae566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:38.828576 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c880fa-a20a-4a8a-aa36-a0d8e1eae566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:38.828859 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c880fa-a20a-4a8a-aa36-a0d8e1eae566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:38.829167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c880fa-a20a-4a8a-aa36-a0d8e1eae566 admin admin] PATH_INFO: `/v3/users/5407539cb9224a5284765103c7c999ca` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:38.830383 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c880fa-a20a-4a8a-aa36-a0d8e1eae566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5407539cb9224a5284765103c7c999ca)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:38.832321 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c880fa-a20a-4a8a-aa36-a0d8e1eae566 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:38.857256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:38.860323 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1128/2247] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:14:38.866917 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c205d47-72c9-44f1-9da0-70495eab36da None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:38.867284 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c205d47-72c9-44f1-9da0-70495eab36da None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:38.867580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c205d47-72c9-44f1-9da0-70495eab36da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:38.891059 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2c205d47-72c9-44f1-9da0-70495eab36da None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:38.892257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2c205d47-72c9-44f1-9da0-70495eab36da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:38.896958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-66c880fa-a20a-4a8a-aa36-a0d8e1eae566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5407539cb9224a5284765103c7c999ca'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:38.902521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1129/2248] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:14:38.906627 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c205d47-72c9-44f1-9da0-70495eab36da None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:38.909116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-66c880fa-a20a-4a8a-aa36-a0d8e1eae566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5407539cb9224a5284765103c7c999ca'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:38.915950 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c205d47-72c9-44f1-9da0-70495eab36da None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:38.921591 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde6591-b3e8-4bd3-9771-e5dd51bf9c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:38.922236 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde6591-b3e8-4bd3-9771-e5dd51bf9c53 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:38.922519 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde6591-b3e8-4bd3-9771-e5dd51bf9c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:38.922789 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bde6591-b3e8-4bd3-9771-e5dd51bf9c53 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:38.923433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde6591-b3e8-4bd3-9771-e5dd51bf9c53 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:14:38.923734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde6591-b3e8-4bd3-9771-e5dd51bf9c53 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:38.925355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bde6591-b3e8-4bd3-9771-e5dd51bf9c53 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:38.926268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-66c880fa-a20a-4a8a-aa36-a0d8e1eae566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5407539cb9224a5284765103c7c999ca'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:38.927237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-66c880fa-a20a-4a8a-aa36-a0d8e1eae566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5407539cb9224a5284765103c7c999ca'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:38.928517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-66c880fa-a20a-4a8a-aa36-a0d8e1eae566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5407539cb9224a5284765103c7c999ca'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:38.939460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1130/2249] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:14:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:14:38.944102 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde6591-b3e8-4bd3-9771-e5dd51bf9c53 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:38.945989 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5bde6591-b3e8-4bd3-9771-e5dd51bf9c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:38.952085 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1120/2250] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:14:38 2022] DELETE /identity/v3/users/5407539cb9224a5284765103c7c999ca => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:14:38.953839 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421fde2-641a-4c9c-a4a0-4b0d8304b0fa None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:38.954223 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421fde2-641a-4c9c-a4a0-4b0d8304b0fa None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:38.954656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421fde2-641a-4c9c-a4a0-4b0d8304b0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:38.970934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bde6591-b3e8-4bd3-9771-e5dd51bf9c53 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:38.977231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4421fde2-641a-4c9c-a4a0-4b0d8304b0fa None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:38.978276 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4421fde2-641a-4c9c-a4a0-4b0d8304b0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:39.023047 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:39.044752 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19ad05-ff73-40ed-a279-24aa872e03cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:39.045463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19ad05-ff73-40ed-a279-24aa872e03cb admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:39.045684 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19ad05-ff73-40ed-a279-24aa872e03cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:39.045958 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19ad05-ff73-40ed-a279-24aa872e03cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:39.046590 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19ad05-ff73-40ed-a279-24aa872e03cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:39.048481 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea19ad05-ff73-40ed-a279-24aa872e03cb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:39.057486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1121/2251] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:14:39.064834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421fde2-641a-4c9c-a4a0-4b0d8304b0fa None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:39.066681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4421fde2-641a-4c9c-a4a0-4b0d8304b0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:39.080403 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1131/2252] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:14:38 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:39.087287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19ad05-ff73-40ed-a279-24aa872e03cb admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:39.141483 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421fde2-641a-4c9c-a4a0-4b0d8304b0fa None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:39.225335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea19ad05-ff73-40ed-a279-24aa872e03cb admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:39.226813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6f2df-acc3-4a4e-9a05-53bf99466288 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:39.227601 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6f2df-acc3-4a4e-9a05-53bf99466288 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:39.228009 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6f2df-acc3-4a4e-9a05-53bf99466288 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:39.228539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d6f2df-acc3-4a4e-9a05-53bf99466288 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:39.246382 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6f2df-acc3-4a4e-9a05-53bf99466288 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:39.249517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d6f2df-acc3-4a4e-9a05-53bf99466288 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:39.257851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e1dfb-52cb-4765-846e-4749b07c2f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:39.258928 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e1dfb-52cb-4765-846e-4749b07c2f3c admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:39.259386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e1dfb-52cb-4765-846e-4749b07c2f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:39.259828 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8e1dfb-52cb-4765-846e-4749b07c2f3c admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:39.260618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e1dfb-52cb-4765-846e-4749b07c2f3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:39.266179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:39.278860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8e1dfb-52cb-4765-846e-4749b07c2f3c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:39.295496 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1122/2253] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:14:39.335397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1132/2254] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:14:39 2022] POST /identity/v3/users => generated 288 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:39.340463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6f2df-acc3-4a4e-9a05-53bf99466288 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:39.353504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d6f2df-acc3-4a4e-9a05-53bf99466288 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:39.367861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6c72c-48d1-4092-be4a-499a3b9e81cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:39.369031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6c72c-48d1-4092-be4a-499a3b9e81cd admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:39.369031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6c72c-48d1-4092-be4a-499a3b9e81cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:39.369389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c6c72c-48d1-4092-be4a-499a3b9e81cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:39.370811 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6c72c-48d1-4092-be4a-499a3b9e81cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:39.376922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c6c72c-48d1-4092-be4a-499a3b9e81cd admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:39.412531 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1123/2255] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:14:39.416442 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e1dfb-52cb-4765-846e-4749b07c2f3c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:39.435853 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8e1dfb-52cb-4765-846e-4749b07c2f3c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:39.457301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4714cd3d-7be7-4e47-8904-dec4df9f7194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:39.457301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714cd3d-7be7-4e47-8904-dec4df9f7194 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:39.457301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714cd3d-7be7-4e47-8904-dec4df9f7194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:39.457301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4714cd3d-7be7-4e47-8904-dec4df9f7194 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:39.457937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714cd3d-7be7-4e47-8904-dec4df9f7194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:39.460090 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4714cd3d-7be7-4e47-8904-dec4df9f7194 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:39.491243 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1133/2256] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:14:39.492116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6c72c-48d1-4092-be4a-499a3b9e81cd admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:39.513068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c6c72c-48d1-4092-be4a-499a3b9e81cd admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:39.526273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4efda-1e31-4d74-a975-6b407316d2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:39.526952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4efda-1e31-4d74-a975-6b407316d2f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:39.527231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4efda-1e31-4d74-a975-6b407316d2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:39.527502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f4efda-1e31-4d74-a975-6b407316d2f3 admin admin] PATH_INFO: `/v3/projects/1fb9cca799f94645b7fbb55e1f8fc8ad/users/842d761d9eba453ba006477f5b8c58db/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:39.567961 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4efda-1e31-4d74-a975-6b407316d2f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fb9cca799f94645b7fbb55e1f8fc8ad, user_id=842d761d9eba453ba006477f5b8c58db, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:39.577015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f4efda-1e31-4d74-a975-6b407316d2f3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:39.620836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1124/2257] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:39 2022] PUT /identity/v3/projects/1fb9cca799f94645b7fbb55e1f8fc8ad/users/842d761d9eba453ba006477f5b8c58db/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:39.632895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92526356-13d6-48c4-b1e3-e1e56fc44d61 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:39.637605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92526356-13d6-48c4-b1e3-e1e56fc44d61 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:39.638041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92526356-13d6-48c4-b1e3-e1e56fc44d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:39.699897 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-92526356-13d6-48c4-b1e3-e1e56fc44d61 None None] MFA Rules not processed for user `842d761d9eba453ba006477f5b8c58db`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:39.701158 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-92526356-13d6-48c4-b1e3-e1e56fc44d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:39.832416 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4efda-1e31-4d74-a975-6b407316d2f3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:39.834258 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-02f4efda-1e31-4d74-a975-6b407316d2f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:39.862993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1134/2258] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:39.867707 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92526356-13d6-48c4-b1e3-e1e56fc44d61 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:39.958745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:39.977159 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f4efda-1e31-4d74-a975-6b407316d2f3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:40.000550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:40.001179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:40.001658 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:40.001793 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42 admin admin] PATH_INFO: `/v3/projects/80f8d7ca20ed4c8bbb167197ddbbfdaf` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:40.002961 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80f8d7ca20ed4c8bbb167197ddbbfdaf)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:40.005127 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:40.017531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92526356-13d6-48c4-b1e3-e1e56fc44d61 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:40.029341 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296aa835-5ef4-4705-ada4-a8128b63ff90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:40.029730 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296aa835-5ef4-4705-ada4-a8128b63ff90 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:40.030013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296aa835-5ef4-4705-ada4-a8128b63ff90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:40.030281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296aa835-5ef4-4705-ada4-a8128b63ff90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:40.031157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296aa835-5ef4-4705-ada4-a8128b63ff90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:40.034458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296aa835-5ef4-4705-ada4-a8128b63ff90 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:40.078881 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1135/2259] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:14:39 2022] POST /identity/v3/projects => generated 377 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:14:40.084272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296aa835-5ef4-4705-ada4-a8128b63ff90 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:40.102584 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80f8d7ca20ed4c8bbb167197ddbbfdaf', 'request_id': 'req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:40.116651 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80f8d7ca20ed4c8bbb167197ddbbfdaf', 'request_id': 'req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:40.125721 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80f8d7ca20ed4c8bbb167197ddbbfdaf', 'request_id': 'req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:40.144105 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42 admin admin] The token cache is being invalidate because project 80f8d7ca20ed4c8bbb167197ddbbfdaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:14:40.144884 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:40.145742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:40.147433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1125/2260] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:14:39 2022] DELETE /identity/v3/projects/80f8d7ca20ed4c8bbb167197ddbbfdaf => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:40.151998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:40.153618 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:40.163235 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296aa835-5ef4-4705-ada4-a8128b63ff90 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:40.218032 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:40.226180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3fa58-d7b9-4d0c-9e28-3b3f93ffc839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:40.226789 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3fa58-d7b9-4d0c-9e28-3b3f93ffc839 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:40.227071 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3fa58-d7b9-4d0c-9e28-3b3f93ffc839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:40.227339 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3fa58-d7b9-4d0c-9e28-3b3f93ffc839 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:40.229099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3fa58-d7b9-4d0c-9e28-3b3f93ffc839 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:40.230851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3fa58-d7b9-4d0c-9e28-3b3f93ffc839 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:40.233139 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe426d-f81f-49c8-a6cb-da48cbc60a42 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:40.238380 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b3807-c064-4f0f-8dce-8e395ccda2a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:40.239076 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b3807-c064-4f0f-8dce-8e395ccda2a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:40.239451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b3807-c064-4f0f-8dce-8e395ccda2a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:40.239724 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b3807-c064-4f0f-8dce-8e395ccda2a0 admin admin] PATH_INFO: `/v3/users/33d5ed55606241258504bb66ac70ae8c` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:40.241305 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b3807-c064-4f0f-8dce-8e395ccda2a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33d5ed55606241258504bb66ac70ae8c)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:40.243255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b3807-c064-4f0f-8dce-8e395ccda2a0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:40.257374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1136/2261] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:14:40 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:14:40.260919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3fa58-d7b9-4d0c-9e28-3b3f93ffc839 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:40.263744 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-50e3fa58-d7b9-4d0c-9e28-3b3f93ffc839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:40.281056 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3fa58-d7b9-4d0c-9e28-3b3f93ffc839 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:40.287570 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60187974-24ff-4a74-9da4-8641309b1a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:40.288182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60187974-24ff-4a74-9da4-8641309b1a9c admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:40.288458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60187974-24ff-4a74-9da4-8641309b1a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:40.288735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60187974-24ff-4a74-9da4-8641309b1a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:40.289408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60187974-24ff-4a74-9da4-8641309b1a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:40.290997 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60187974-24ff-4a74-9da4-8641309b1a9c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:40.302146 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1137/2262] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:14:40.306085 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60187974-24ff-4a74-9da4-8641309b1a9c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:40.309709 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-798b3807-c064-4f0f-8dce-8e395ccda2a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33d5ed55606241258504bb66ac70ae8c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:40.316720 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60187974-24ff-4a74-9da4-8641309b1a9c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:40.317532 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-798b3807-c064-4f0f-8dce-8e395ccda2a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33d5ed55606241258504bb66ac70ae8c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:40.321640 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e095f0-bd8a-4781-93aa-324c4257b770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:40.322222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e095f0-bd8a-4781-93aa-324c4257b770 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:40.322550 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e095f0-bd8a-4781-93aa-324c4257b770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:40.322966 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e095f0-bd8a-4781-93aa-324c4257b770 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:40.323598 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e095f0-bd8a-4781-93aa-324c4257b770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:40.325173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e095f0-bd8a-4781-93aa-324c4257b770 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:40.330724 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-798b3807-c064-4f0f-8dce-8e395ccda2a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33d5ed55606241258504bb66ac70ae8c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:40.331439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-798b3807-c064-4f0f-8dce-8e395ccda2a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33d5ed55606241258504bb66ac70ae8c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:40.332063 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-798b3807-c064-4f0f-8dce-8e395ccda2a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33d5ed55606241258504bb66ac70ae8c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:40.337243 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1138/2263] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:14:40.341447 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e095f0-bd8a-4781-93aa-324c4257b770 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:40.343357 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-82e095f0-bd8a-4781-93aa-324c4257b770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:40.351553 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1126/2264] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:14:40 2022] DELETE /identity/v3/users/33d5ed55606241258504bb66ac70ae8c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:40.358838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e095f0-bd8a-4781-93aa-324c4257b770 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:40.394719 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:40.436675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5db63d-ec60-4e67-a56e-a129441411b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:40.437686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5db63d-ec60-4e67-a56e-a129441411b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:40.438168 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5db63d-ec60-4e67-a56e-a129441411b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:40.438983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5db63d-ec60-4e67-a56e-a129441411b3 admin admin] PATH_INFO: `/v3/projects/c3a6f580e64f40178a4d7bd571e4d516/users/6975c49fc5dc4b2ab388c89a4cb93ae2/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:40.469013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5db63d-ec60-4e67-a56e-a129441411b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a6f580e64f40178a4d7bd571e4d516, user_id=6975c49fc5dc4b2ab388c89a4cb93ae2, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:40.472983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5db63d-ec60-4e67-a56e-a129441411b3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:40.490214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1139/2265] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:40 2022] PUT /identity/v3/projects/c3a6f580e64f40178a4d7bd571e4d516/users/6975c49fc5dc4b2ab388c89a4cb93ae2/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:40.500759 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887f4e3-556f-4eab-acab-3613ea86a53b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:40.501052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887f4e3-556f-4eab-acab-3613ea86a53b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:40.501355 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b887f4e3-556f-4eab-acab-3613ea86a53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:40.530492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b887f4e3-556f-4eab-acab-3613ea86a53b None None] MFA Rules not processed for user `6975c49fc5dc4b2ab388c89a4cb93ae2`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:40.531721 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b887f4e3-556f-4eab-acab-3613ea86a53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:40.615748 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1127/2266] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:40.617020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5db63d-ec60-4e67-a56e-a129441411b3 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:40.728298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5db63d-ec60-4e67-a56e-a129441411b3 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:40.734642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4da7c0-c436-4101-8525-61db67da1912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:40.735367 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4da7c0-c436-4101-8525-61db67da1912 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:40.735718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4da7c0-c436-4101-8525-61db67da1912 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:40.736089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4da7c0-c436-4101-8525-61db67da1912 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:40.736815 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4da7c0-c436-4101-8525-61db67da1912 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:40.738902 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4da7c0-c436-4101-8525-61db67da1912 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:40.769222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887f4e3-556f-4eab-acab-3613ea86a53b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:40.772046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b887f4e3-556f-4eab-acab-3613ea86a53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:40.795778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1140/2267] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:14:40 2022] POST /identity/v3/projects => generated 375 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:40.802422 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4da7c0-c436-4101-8525-61db67da1912 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:40.804443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b887f4e3-556f-4eab-acab-3613ea86a53b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:40.870006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b484b-170c-45dd-887d-a127ad804588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:40.870757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b484b-170c-45dd-887d-a127ad804588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:40.871255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b484b-170c-45dd-887d-a127ad804588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:40.871533 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b484b-170c-45dd-887d-a127ad804588 admin admin] PATH_INFO: `/v3/projects/cfe1d38f14674e6da2f685b10ee0361b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:40.872708 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b484b-170c-45dd-887d-a127ad804588 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfe1d38f14674e6da2f685b10ee0361b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:40.874839 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b484b-170c-45dd-887d-a127ad804588 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:40.878079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:40.892916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4da7c0-c436-4101-8525-61db67da1912 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:40.900131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4cd2cc-ff0b-4dba-b485-f55ed78a909f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:40.901593 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4cd2cc-ff0b-4dba-b485-f55ed78a909f admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:40.901870 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4cd2cc-ff0b-4dba-b485-f55ed78a909f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:40.902215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4cd2cc-ff0b-4dba-b485-f55ed78a909f admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:40.902859 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4cd2cc-ff0b-4dba-b485-f55ed78a909f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:40.905523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4cd2cc-ff0b-4dba-b485-f55ed78a909f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:40.937362 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1141/2268] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:14:40 2022] POST /identity/v3/users => generated 294 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:40.939604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4cd2cc-ff0b-4dba-b485-f55ed78a909f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:40.951170 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4cd2cc-ff0b-4dba-b485-f55ed78a909f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:40.954477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b4b484b-170c-45dd-887d-a127ad804588 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfe1d38f14674e6da2f685b10ee0361b', 'request_id': 'req-0b4b484b-170c-45dd-887d-a127ad804588'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:40.965294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b4b484b-170c-45dd-887d-a127ad804588 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfe1d38f14674e6da2f685b10ee0361b', 'request_id': 'req-0b4b484b-170c-45dd-887d-a127ad804588'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:40.985384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b4b484b-170c-45dd-887d-a127ad804588 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfe1d38f14674e6da2f685b10ee0361b', 'request_id': 'req-0b4b484b-170c-45dd-887d-a127ad804588'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:40.993155 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:41.000190 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b4b484b-170c-45dd-887d-a127ad804588 admin admin] The token cache is being invalidate because project cfe1d38f14674e6da2f685b10ee0361b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:14:41.000506 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b4b484b-170c-45dd-887d-a127ad804588 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:41.001177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b4b484b-170c-45dd-887d-a127ad804588 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:41.003137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1128/2269] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:14:40 2022] DELETE /identity/v3/projects/cfe1d38f14674e6da2f685b10ee0361b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:41.009002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1e096-30e7-444d-b4e5-c564d5342753 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:41.010605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1e096-30e7-444d-b4e5-c564d5342753 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:41.011804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda1e096-30e7-444d-b4e5-c564d5342753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:41.026584 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88127da2-7cfd-4a23-a80d-679f78cde1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:41.027197 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88127da2-7cfd-4a23-a80d-679f78cde1be admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:41.027473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88127da2-7cfd-4a23-a80d-679f78cde1be admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:41.027743 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88127da2-7cfd-4a23-a80d-679f78cde1be admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:41.028389 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88127da2-7cfd-4a23-a80d-679f78cde1be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:41.030108 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88127da2-7cfd-4a23-a80d-679f78cde1be admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:41.041403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-eda1e096-30e7-444d-b4e5-c564d5342753 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:41.042823 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-eda1e096-30e7-444d-b4e5-c564d5342753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:41.044102 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1142/2270] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:14:41.045136 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88127da2-7cfd-4a23-a80d-679f78cde1be admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:41.047521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-88127da2-7cfd-4a23-a80d-679f78cde1be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:41.051664 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1129/2271] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:14:41.059516 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5eadc-15ad-4f84-a472-fddc0d30b91a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:41.061919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5eadc-15ad-4f84-a472-fddc0d30b91a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:41.061919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5eadc-15ad-4f84-a472-fddc0d30b91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:41.063382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88127da2-7cfd-4a23-a80d-679f78cde1be admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:41.072131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4a71e-0742-4037-9996-2aab7b7c1956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:41.074715 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4a71e-0742-4037-9996-2aab7b7c1956 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:41.075613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4a71e-0742-4037-9996-2aab7b7c1956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:41.076494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e4a71e-0742-4037-9996-2aab7b7c1956 admin admin] PATH_INFO: `/v3/projects/df60022cfb934d5095ee645fd498006f/users/b107cf83f7ac4b91a63c665ff86890ee/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:41.084097 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a0f5eadc-15ad-4f84-a472-fddc0d30b91a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:41.086271 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a0f5eadc-15ad-4f84-a472-fddc0d30b91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:41.100743 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1130/2272] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:14:41.101900 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4a71e-0742-4037-9996-2aab7b7c1956 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df60022cfb934d5095ee645fd498006f, user_id=b107cf83f7ac4b91a63c665ff86890ee, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:41.102542 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5eadc-15ad-4f84-a472-fddc0d30b91a None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:41.106642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e4a71e-0742-4037-9996-2aab7b7c1956 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:41.113288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5eadc-15ad-4f84-a472-fddc0d30b91a None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:41.118487 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32962f69-1c56-4732-ab7d-65be04e237c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:41.119218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32962f69-1c56-4732-ab7d-65be04e237c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:41.119474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32962f69-1c56-4732-ab7d-65be04e237c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:41.119795 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32962f69-1c56-4732-ab7d-65be04e237c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:41.120554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32962f69-1c56-4732-ab7d-65be04e237c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:14:41.120904 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32962f69-1c56-4732-ab7d-65be04e237c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:41.122526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32962f69-1c56-4732-ab7d-65be04e237c3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:41.128896 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1143/2273] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:41 2022] PUT /identity/v3/projects/df60022cfb934d5095ee645fd498006f/users/b107cf83f7ac4b91a63c665ff86890ee/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:14:41.134245 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4a71e-0742-4037-9996-2aab7b7c1956 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:41.139959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1131/2274] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:14:41.141007 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32962f69-1c56-4732-ab7d-65be04e237c3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:41.154818 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32962f69-1c56-4732-ab7d-65be04e237c3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:41.228752 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e4a71e-0742-4037-9996-2aab7b7c1956 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:41.235476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdd024-277a-4fc9-8949-c26516522706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:41.236534 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdd024-277a-4fc9-8949-c26516522706 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:41.236819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdd024-277a-4fc9-8949-c26516522706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:41.237596 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbdd024-277a-4fc9-8949-c26516522706 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:41.238259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbdd024-277a-4fc9-8949-c26516522706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:41.240894 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbdd024-277a-4fc9-8949-c26516522706 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:41.251450 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936b35f-a9bc-4af6-9525-85f092650a67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:41.252349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936b35f-a9bc-4af6-9525-85f092650a67 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:41.253081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936b35f-a9bc-4af6-9525-85f092650a67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:41.253571 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6936b35f-a9bc-4af6-9525-85f092650a67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:41.254511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936b35f-a9bc-4af6-9525-85f092650a67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:41.257084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6936b35f-a9bc-4af6-9525-85f092650a67 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:41.257570 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1144/2275] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:14:41.258474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdd024-277a-4fc9-8949-c26516522706 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:41.276943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbdd024-277a-4fc9-8949-c26516522706 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:41.282559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35f7d9c-1585-4d33-940d-867db892d81e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:41.284106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f7d9c-1585-4d33-940d-867db892d81e admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:41.284675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f7d9c-1585-4d33-940d-867db892d81e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:41.285174 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35f7d9c-1585-4d33-940d-867db892d81e admin admin] PATH_INFO: `/v3/projects/df60022cfb934d5095ee645fd498006f/users/b107cf83f7ac4b91a63c665ff86890ee/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:41.293428 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f7d9c-1585-4d33-940d-867db892d81e admin admin] RBAC: Authorizing `identity:create_grant(project_id=df60022cfb934d5095ee645fd498006f, user_id=b107cf83f7ac4b91a63c665ff86890ee, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:41.296311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35f7d9c-1585-4d33-940d-867db892d81e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:41.318272 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1132/2276] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:14:41 2022] POST /identity/v3/projects => generated 391 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:41.319324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936b35f-a9bc-4af6-9525-85f092650a67 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:41.330605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1145/2277] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:41 2022] PUT /identity/v3/projects/df60022cfb934d5095ee645fd498006f/users/b107cf83f7ac4b91a63c665ff86890ee/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:41.337840 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ebbdb-de16-4a5a-a6b2-f4d882aaa263 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:41.338725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ebbdb-de16-4a5a-a6b2-f4d882aaa263 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:41.339245 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50ebbdb-de16-4a5a-a6b2-f4d882aaa263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:41.388457 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e50ebbdb-de16-4a5a-a6b2-f4d882aaa263 None None] MFA Rules not processed for user `b107cf83f7ac4b91a63c665ff86890ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:41.391011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e50ebbdb-de16-4a5a-a6b2-f4d882aaa263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:41.402517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:41.419126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6936b35f-a9bc-4af6-9525-85f092650a67 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:41.427186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72947c9-91ff-4878-9a9b-2ebc12eca635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:41.429565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72947c9-91ff-4878-9a9b-2ebc12eca635 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:41.430103 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72947c9-91ff-4878-9a9b-2ebc12eca635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:41.430609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72947c9-91ff-4878-9a9b-2ebc12eca635 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:41.433282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72947c9-91ff-4878-9a9b-2ebc12eca635 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:41.435369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72947c9-91ff-4878-9a9b-2ebc12eca635 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:41.470523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1133/2278] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:14:41 2022] POST /identity/v3/users => generated 296 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:41.472315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72947c9-91ff-4878-9a9b-2ebc12eca635 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:41.485998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72947c9-91ff-4878-9a9b-2ebc12eca635 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:41.490784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1119abb-06ce-4532-9e46-6c99df89ac05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:41.491488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1119abb-06ce-4532-9e46-6c99df89ac05 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:41.491890 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1119abb-06ce-4532-9e46-6c99df89ac05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:41.492260 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1119abb-06ce-4532-9e46-6c99df89ac05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:41.493008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1119abb-06ce-4532-9e46-6c99df89ac05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:41.495402 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1119abb-06ce-4532-9e46-6c99df89ac05 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:41.508979 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1146/2279] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:41.518106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1134/2280] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:14:41.523583 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ebbdb-de16-4a5a-a6b2-f4d882aaa263 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:41.526787 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1119abb-06ce-4532-9e46-6c99df89ac05 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:41.529132 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a1119abb-06ce-4532-9e46-6c99df89ac05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:41.542864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50ebbdb-de16-4a5a-a6b2-f4d882aaa263 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:41.548747 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2de0b4-aab3-414e-9187-94898425f477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:41.549387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2de0b4-aab3-414e-9187-94898425f477 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:41.552049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2de0b4-aab3-414e-9187-94898425f477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:41.552358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2de0b4-aab3-414e-9187-94898425f477 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:41.553048 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2de0b4-aab3-414e-9187-94898425f477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:41.555037 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2de0b4-aab3-414e-9187-94898425f477 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:41.571621 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1147/2281] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:14:41.575940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2de0b4-aab3-414e-9187-94898425f477 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:41.583186 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:41.589077 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2de0b4-aab3-414e-9187-94898425f477 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:41.597025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b7ba85-15fb-4a0a-bd8b-121dc3270d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:41.597728 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7ba85-15fb-4a0a-bd8b-121dc3270d4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:41.598009 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7ba85-15fb-4a0a-bd8b-121dc3270d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:41.598285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b7ba85-15fb-4a0a-bd8b-121dc3270d4c admin admin] PATH_INFO: `/v3/projects/024984cafe084754ac3544c7a40a1773/users/1ff06b2d2aa94ede8619b6a895738e97/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:41.612333 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1119abb-06ce-4532-9e46-6c99df89ac05 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:41.616496 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66456756-a54f-4376-8d42-cbc2cd47fd0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:41.617377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66456756-a54f-4376-8d42-cbc2cd47fd0e service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:41.617854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66456756-a54f-4376-8d42-cbc2cd47fd0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:41.618311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66456756-a54f-4376-8d42-cbc2cd47fd0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:41.620180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7ba85-15fb-4a0a-bd8b-121dc3270d4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=024984cafe084754ac3544c7a40a1773, user_id=1ff06b2d2aa94ede8619b6a895738e97, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:41.622834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-66456756-a54f-4376-8d42-cbc2cd47fd0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:41.623550 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b7ba85-15fb-4a0a-bd8b-121dc3270d4c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:41.649744 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1148/2282] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:41 2022] PUT /identity/v3/projects/024984cafe084754ac3544c7a40a1773/users/1ff06b2d2aa94ede8619b6a895738e97/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:14:41.657603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3194d-98b1-4111-9e22-b7d475d28d52 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:41.657880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3194d-98b1-4111-9e22-b7d475d28d52 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:41.658147 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3194d-98b1-4111-9e22-b7d475d28d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:41.693628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4bf3194d-98b1-4111-9e22-b7d475d28d52 None None] MFA Rules not processed for user `1ff06b2d2aa94ede8619b6a895738e97`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:41.693914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4bf3194d-98b1-4111-9e22-b7d475d28d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:41.698789 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66456756-a54f-4376-8d42-cbc2cd47fd0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:41.700620 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66456756-a54f-4376-8d42-cbc2cd47fd0e service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:41.711278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1135/2283] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:14:41.776455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1149/2284] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:14:41.780985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66456756-a54f-4376-8d42-cbc2cd47fd0e service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:41.792836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66456756-a54f-4376-8d42-cbc2cd47fd0e service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:41.906006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b6a64-4273-4fdc-a38e-d0e80661a446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:41.907509 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b6a64-4273-4fdc-a38e-d0e80661a446 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:41.908000 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b6a64-4273-4fdc-a38e-d0e80661a446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:41.908488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742b6a64-4273-4fdc-a38e-d0e80661a446 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:41.909995 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b6a64-4273-4fdc-a38e-d0e80661a446 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:41.912013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742b6a64-4273-4fdc-a38e-d0e80661a446 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:41.955241 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1136/2285] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:14:41 2022] POST /identity/v3/projects => generated 391 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:41.959557 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3194d-98b1-4111-9e22-b7d475d28d52 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:41.985006 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf3194d-98b1-4111-9e22-b7d475d28d52 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:42.071101 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebec2b-f558-482d-a0c1-5f486a0b8e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:42.071722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebec2b-f558-482d-a0c1-5f486a0b8e92 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:42.072047 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebec2b-f558-482d-a0c1-5f486a0b8e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:42.072336 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ebec2b-f558-482d-a0c1-5f486a0b8e92 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:42.073508 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ebec2b-f558-482d-a0c1-5f486a0b8e92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:42.075600 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ebec2b-f558-482d-a0c1-5f486a0b8e92 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:42.110180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1150/2286] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:14:41 2022] POST /identity/v3/users => generated 302 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:42.113964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b6a64-4273-4fdc-a38e-d0e80661a446 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:42.127346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742b6a64-4273-4fdc-a38e-d0e80661a446 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:42.142258 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc8201-8f2e-44cc-ac54-2ea04f0e9df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:42.142858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc8201-8f2e-44cc-ac54-2ea04f0e9df0 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:42.143925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc8201-8f2e-44cc-ac54-2ea04f0e9df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:42.143925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc8201-8f2e-44cc-ac54-2ea04f0e9df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:42.144114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc8201-8f2e-44cc-ac54-2ea04f0e9df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:42.145746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc8201-8f2e-44cc-ac54-2ea04f0e9df0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:42.149371 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:42.165227 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1137/2287] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:14:42.166504 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebec2b-f558-482d-a0c1-5f486a0b8e92 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:42.179012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ebec2b-f558-482d-a0c1-5f486a0b8e92 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:42.190594 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91358d-c78a-4d59-b7d2-bef6b1ef3397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:42.191478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91358d-c78a-4d59-b7d2-bef6b1ef3397 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:42.191924 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91358d-c78a-4d59-b7d2-bef6b1ef3397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:42.194096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe91358d-c78a-4d59-b7d2-bef6b1ef3397 admin admin] PATH_INFO: `/v3/projects/4c2981493b4a40719c4f8aa3e55b577f/users/8f79b072567046da8531576e60cf5876/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:42.234825 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91358d-c78a-4d59-b7d2-bef6b1ef3397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c2981493b4a40719c4f8aa3e55b577f, user_id=8f79b072567046da8531576e60cf5876, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:42.239650 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe91358d-c78a-4d59-b7d2-bef6b1ef3397 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:42.283303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1151/2288] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:42 2022] PUT /identity/v3/projects/4c2981493b4a40719c4f8aa3e55b577f/users/8f79b072567046da8531576e60cf5876/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:42.283462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc8201-8f2e-44cc-ac54-2ea04f0e9df0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:42.298798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc8201-8f2e-44cc-ac54-2ea04f0e9df0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:42.387267 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5099712-c2dd-4ab8-bfcf-3069b1c2c687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:42.387879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5099712-c2dd-4ab8-bfcf-3069b1c2c687 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:42.388216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5099712-c2dd-4ab8-bfcf-3069b1c2c687 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:42.388636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5099712-c2dd-4ab8-bfcf-3069b1c2c687 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:42.389368 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5099712-c2dd-4ab8-bfcf-3069b1c2c687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:42.391370 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5099712-c2dd-4ab8-bfcf-3069b1c2c687 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:42.402988 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1138/2289] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:14:42.406466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91358d-c78a-4d59-b7d2-bef6b1ef3397 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:42.417694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe91358d-c78a-4d59-b7d2-bef6b1ef3397 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:42.423233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c4ae2-74fa-4381-8757-cce8471d5117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:42.423401 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c4ae2-74fa-4381-8757-cce8471d5117 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:42.423613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c4ae2-74fa-4381-8757-cce8471d5117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:42.424197 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344c4ae2-74fa-4381-8757-cce8471d5117 admin admin] PATH_INFO: `/v3/projects/4c2981493b4a40719c4f8aa3e55b577f/users/8f79b072567046da8531576e60cf5876/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:42.426150 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c4ae2-74fa-4381-8757-cce8471d5117 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c2981493b4a40719c4f8aa3e55b577f, user_id=8f79b072567046da8531576e60cf5876, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:42.428853 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344c4ae2-74fa-4381-8757-cce8471d5117 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:42.442286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1152/2290] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:42 2022] PUT /identity/v3/projects/4c2981493b4a40719c4f8aa3e55b577f/users/8f79b072567046da8531576e60cf5876/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:42.453999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593b786-7d06-4e41-a1d1-921e4e1bcdb4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:42.453999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593b786-7d06-4e41-a1d1-921e4e1bcdb4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:42.453999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e593b786-7d06-4e41-a1d1-921e4e1bcdb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:42.485042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e593b786-7d06-4e41-a1d1-921e4e1bcdb4 None None] MFA Rules not processed for user `8f79b072567046da8531576e60cf5876`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:42.490426 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e593b786-7d06-4e41-a1d1-921e4e1bcdb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:42.592298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1139/2291] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:42 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:42.614856 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c4ae2-74fa-4381-8757-cce8471d5117 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:42.617352 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-344c4ae2-74fa-4381-8757-cce8471d5117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:42.649777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:42.701740 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344c4ae2-74fa-4381-8757-cce8471d5117 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:42.706415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11d18c-cde1-45c6-b9f5-e433a294bcd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:42.707089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11d18c-cde1-45c6-b9f5-e433a294bcd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:42.707366 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11d18c-cde1-45c6-b9f5-e433a294bcd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:42.707633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c11d18c-cde1-45c6-b9f5-e433a294bcd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:42.722697 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11d18c-cde1-45c6-b9f5-e433a294bcd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:42.724627 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c11d18c-cde1-45c6-b9f5-e433a294bcd8 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:42.736544 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1153/2292] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:42 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:14:42.790916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593b786-7d06-4e41-a1d1-921e4e1bcdb4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:42.802748 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e593b786-7d06-4e41-a1d1-921e4e1bcdb4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:42.806878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecf12f-5655-44ac-9c65-0e25050920a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:42.808354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecf12f-5655-44ac-9c65-0e25050920a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:42.808635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecf12f-5655-44ac-9c65-0e25050920a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:42.811467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ecf12f-5655-44ac-9c65-0e25050920a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:42.893168 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ecf12f-5655-44ac-9c65-0e25050920a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:42.894818 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ecf12f-5655-44ac-9c65-0e25050920a9 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:42.906252 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1140/2293] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:14:43.574031 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11d18c-cde1-45c6-b9f5-e433a294bcd8 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:43.583928 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c11d18c-cde1-45c6-b9f5-e433a294bcd8 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:43.587843 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e4472-c56a-4b49-88df-925b32fbb67f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:43.588521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e4472-c56a-4b49-88df-925b32fbb67f service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:43.589232 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e4472-c56a-4b49-88df-925b32fbb67f service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:43.589232 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e4472-c56a-4b49-88df-925b32fbb67f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:43.590884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e69e4472-c56a-4b49-88df-925b32fbb67f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:43.700066 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e4472-c56a-4b49-88df-925b32fbb67f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:43.701577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e4472-c56a-4b49-88df-925b32fbb67f service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:43.715036 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1154/2294] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:43 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:14:44.463925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecf12f-5655-44ac-9c65-0e25050920a9 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:44.472505 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ecf12f-5655-44ac-9c65-0e25050920a9 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:44.477061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c43d5-b43c-4b31-952f-3b8a37931d8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:44.477906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c43d5-b43c-4b31-952f-3b8a37931d8e service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:44.478179 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c43d5-b43c-4b31-952f-3b8a37931d8e service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:44.478960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c43d5-b43c-4b31-952f-3b8a37931d8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:44.498704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:44.557760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c43d5-b43c-4b31-952f-3b8a37931d8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:44.559351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c43d5-b43c-4b31-952f-3b8a37931d8e service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:44.569150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1141/2295] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:14:44.843863 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad56e8b-4d97-48c5-b3ca-1cbfe5ca8009 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:44.844370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad56e8b-4d97-48c5-b3ca-1cbfe5ca8009 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:44.844370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad56e8b-4d97-48c5-b3ca-1cbfe5ca8009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:44.865086 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-dad56e8b-4d97-48c5-b3ca-1cbfe5ca8009 None None] MFA Rules not processed for user `e33c120ef02f4f6e8bea53107d90034b`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:44.866377 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-dad56e8b-4d97-48c5-b3ca-1cbfe5ca8009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:44.924769 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1155/2296] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:45.687189 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c43d5-b43c-4b31-952f-3b8a37931d8e service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:45.689178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-dd8c43d5-b43c-4b31-952f-3b8a37931d8e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:45.832555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c43d5-b43c-4b31-952f-3b8a37931d8e service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:45.838756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f815f26b-72bc-418c-8f14-3d216f4a8b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:45.839389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815f26b-72bc-418c-8f14-3d216f4a8b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:45.839827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815f26b-72bc-418c-8f14-3d216f4a8b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:45.840264 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f815f26b-72bc-418c-8f14-3d216f4a8b41 admin admin] PATH_INFO: `/v3/users/b4869c70eac84fddb30090585bc1f0de` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:45.841625 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815f26b-72bc-418c-8f14-3d216f4a8b41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4869c70eac84fddb30090585bc1f0de)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:45.844161 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f815f26b-72bc-418c-8f14-3d216f4a8b41 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:45.938094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f815f26b-72bc-418c-8f14-3d216f4a8b41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4869c70eac84fddb30090585bc1f0de'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:45.947650 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f815f26b-72bc-418c-8f14-3d216f4a8b41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4869c70eac84fddb30090585bc1f0de'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:45.962387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f815f26b-72bc-418c-8f14-3d216f4a8b41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4869c70eac84fddb30090585bc1f0de'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:45.963816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f815f26b-72bc-418c-8f14-3d216f4a8b41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4869c70eac84fddb30090585bc1f0de'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:45.964613 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f815f26b-72bc-418c-8f14-3d216f4a8b41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4869c70eac84fddb30090585bc1f0de'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:45.978325 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1142/2297] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:14:45 2022] DELETE /identity/v3/users/b4869c70eac84fddb30090585bc1f0de => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:46.239704 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad56e8b-4d97-48c5-b3ca-1cbfe5ca8009 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:46.261721 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad56e8b-4d97-48c5-b3ca-1cbfe5ca8009 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:46.328412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7435b1-ba00-4a5a-ba0c-ad232902442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:46.329046 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7435b1-ba00-4a5a-ba0c-ad232902442b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:46.329376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7435b1-ba00-4a5a-ba0c-ad232902442b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:46.329648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7435b1-ba00-4a5a-ba0c-ad232902442b admin admin] PATH_INFO: `/v3/projects/ba8782e8b0af4a2eb7335a0500716b52` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:46.330760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7435b1-ba00-4a5a-ba0c-ad232902442b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba8782e8b0af4a2eb7335a0500716b52)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:46.332648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7435b1-ba00-4a5a-ba0c-ad232902442b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:46.344114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:46.382772 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ac7435b1-ba00-4a5a-ba0c-ad232902442b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba8782e8b0af4a2eb7335a0500716b52', 'request_id': 'req-ac7435b1-ba00-4a5a-ba0c-ad232902442b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:46.391320 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ac7435b1-ba00-4a5a-ba0c-ad232902442b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba8782e8b0af4a2eb7335a0500716b52', 'request_id': 'req-ac7435b1-ba00-4a5a-ba0c-ad232902442b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:46.399067 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ac7435b1-ba00-4a5a-ba0c-ad232902442b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba8782e8b0af4a2eb7335a0500716b52', 'request_id': 'req-ac7435b1-ba00-4a5a-ba0c-ad232902442b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:46.411828 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ac7435b1-ba00-4a5a-ba0c-ad232902442b admin admin] The token cache is being invalidate because project ba8782e8b0af4a2eb7335a0500716b52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:14:46.412148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ac7435b1-ba00-4a5a-ba0c-ad232902442b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:46.412752 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ac7435b1-ba00-4a5a-ba0c-ad232902442b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:46.414305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1156/2298] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:14:46 2022] DELETE /identity/v3/projects/ba8782e8b0af4a2eb7335a0500716b52 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:46.420060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b33ae-c822-4bc6-ad2d-de4b14e33dfa None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:46.420343 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b33ae-c822-4bc6-ad2d-de4b14e33dfa None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:46.420609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3b33ae-c822-4bc6-ad2d-de4b14e33dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:46.440698 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9e3b33ae-c822-4bc6-ad2d-de4b14e33dfa None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:46.442084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9e3b33ae-c822-4bc6-ad2d-de4b14e33dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:46.472857 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:46.527675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1143/2299] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:46.536129 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a54ff-0288-4aa1-a60a-3382a9fa094c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:46.536592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a54ff-0288-4aa1-a60a-3382a9fa094c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:46.537031 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465a54ff-0288-4aa1-a60a-3382a9fa094c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:46.567433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-465a54ff-0288-4aa1-a60a-3382a9fa094c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:46.569700 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-465a54ff-0288-4aa1-a60a-3382a9fa094c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:46.578204 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1157/2300] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:46.582881 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b33ae-c822-4bc6-ad2d-de4b14e33dfa None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:46.595637 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3b33ae-c822-4bc6-ad2d-de4b14e33dfa None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:46.609091 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2a07e-f556-480c-a1a4-761fc0d08788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:46.609716 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2a07e-f556-480c-a1a4-761fc0d08788 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:46.609921 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2a07e-f556-480c-a1a4-761fc0d08788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:46.610189 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb2a07e-f556-480c-a1a4-761fc0d08788 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:46.610828 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2a07e-f556-480c-a1a4-761fc0d08788 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:14:46.611129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2a07e-f556-480c-a1a4-761fc0d08788 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:46.612698 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb2a07e-f556-480c-a1a4-761fc0d08788 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:46.629232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1144/2301] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:14:46.632623 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a54ff-0288-4aa1-a60a-3382a9fa094c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:46.642926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465a54ff-0288-4aa1-a60a-3382a9fa094c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:46.647831 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487bf59-8ba2-44d8-a974-09bc2036086c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:46.648428 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487bf59-8ba2-44d8-a974-09bc2036086c admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:46.648708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487bf59-8ba2-44d8-a974-09bc2036086c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:46.648979 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487bf59-8ba2-44d8-a974-09bc2036086c admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:46.649647 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487bf59-8ba2-44d8-a974-09bc2036086c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:46.652249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487bf59-8ba2-44d8-a974-09bc2036086c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:46.686108 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1158/2302] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:14:46 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:46.690470 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2a07e-f556-480c-a1a4-761fc0d08788 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:46.700877 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb2a07e-f556-480c-a1a4-761fc0d08788 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:46.762786 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ccfa0-edf7-447a-9de2-3a047ee5e466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:46.763389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ccfa0-edf7-447a-9de2-3a047ee5e466 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:46.763660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ccfa0-edf7-447a-9de2-3a047ee5e466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:46.763927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ccfa0-edf7-447a-9de2-3a047ee5e466 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:46.764573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ccfa0-edf7-447a-9de2-3a047ee5e466 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:46.766480 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ccfa0-edf7-447a-9de2-3a047ee5e466 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:46.795509 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1145/2303] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:14:46 2022] POST /identity/v3/users => generated 284 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:14:46.799674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487bf59-8ba2-44d8-a974-09bc2036086c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:46.811317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487bf59-8ba2-44d8-a974-09bc2036086c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:46.816146 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bcb6e-6a3f-4095-9d00-083c3a7a58c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:46.816751 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bcb6e-6a3f-4095-9d00-083c3a7a58c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:46.817027 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bcb6e-6a3f-4095-9d00-083c3a7a58c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:46.817396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78bcb6e-6a3f-4095-9d00-083c3a7a58c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:46.818034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bcb6e-6a3f-4095-9d00-083c3a7a58c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:46.819616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78bcb6e-6a3f-4095-9d00-083c3a7a58c8 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:46.833876 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1159/2304] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:14:46.837832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ccfa0-edf7-447a-9de2-3a047ee5e466 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:46.847879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ccfa0-edf7-447a-9de2-3a047ee5e466 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:46.852517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853d403d-f273-496e-ad4a-098cb26a6b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:46.853237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d403d-f273-496e-ad4a-098cb26a6b48 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:46.853534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d403d-f273-496e-ad4a-098cb26a6b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:46.853806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853d403d-f273-496e-ad4a-098cb26a6b48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:46.854471 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d403d-f273-496e-ad4a-098cb26a6b48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:46.856072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853d403d-f273-496e-ad4a-098cb26a6b48 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:46.866824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1146/2305] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:14:46.871494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bcb6e-6a3f-4095-9d00-083c3a7a58c8 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:46.882085 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78bcb6e-6a3f-4095-9d00-083c3a7a58c8 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:46.887559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02d395-b7b7-4d70-a8b0-92f4203e5964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:46.888215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02d395-b7b7-4d70-a8b0-92f4203e5964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:46.888498 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02d395-b7b7-4d70-a8b0-92f4203e5964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:46.888766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae02d395-b7b7-4d70-a8b0-92f4203e5964 admin admin] PATH_INFO: `/v3/projects/a2ed52b48e6f4c28880ebe8e3b0cb17c/users/19774677040b44a0adff88b6563b8ff9/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:46.904575 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02d395-b7b7-4d70-a8b0-92f4203e5964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ed52b48e6f4c28880ebe8e3b0cb17c, user_id=19774677040b44a0adff88b6563b8ff9, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:46.907322 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae02d395-b7b7-4d70-a8b0-92f4203e5964 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:46.921371 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1160/2306] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:46 2022] PUT /identity/v3/projects/a2ed52b48e6f4c28880ebe8e3b0cb17c/users/19774677040b44a0adff88b6563b8ff9/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:46.933181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8c00f-6326-4370-83fb-c13ec107bcf4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:46.933480 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8c00f-6326-4370-83fb-c13ec107bcf4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:46.933750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee8c00f-6326-4370-83fb-c13ec107bcf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:46.962198 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-eee8c00f-6326-4370-83fb-c13ec107bcf4 None None] MFA Rules not processed for user `19774677040b44a0adff88b6563b8ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:46.963470 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-eee8c00f-6326-4370-83fb-c13ec107bcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:47.030931 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1147/2307] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:47.043710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02d395-b7b7-4d70-a8b0-92f4203e5964 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:47.046321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ae02d395-b7b7-4d70-a8b0-92f4203e5964 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:47.112814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae02d395-b7b7-4d70-a8b0-92f4203e5964 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:47.116644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac388d5-225e-4405-b831-5fdf1d66c87b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:47.117294 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac388d5-225e-4405-b831-5fdf1d66c87b service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:47.117576 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac388d5-225e-4405-b831-5fdf1d66c87b service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:47.117849 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac388d5-225e-4405-b831-5fdf1d66c87b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:47.128693 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac388d5-225e-4405-b831-5fdf1d66c87b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:47.130491 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac388d5-225e-4405-b831-5fdf1d66c87b service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:47.142080 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1161/2308] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:14:47.830286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8c00f-6326-4370-83fb-c13ec107bcf4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:47.831730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-eee8c00f-6326-4370-83fb-c13ec107bcf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:47.863137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:47.899059 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee8c00f-6326-4370-83fb-c13ec107bcf4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:47.904211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb9741-f731-46e0-86ad-6f9821957fba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:47.904851 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb9741-f731-46e0-86ad-6f9821957fba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:47.905124 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb9741-f731-46e0-86ad-6f9821957fba admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:47.905438 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb9741-f731-46e0-86ad-6f9821957fba admin admin] PATH_INFO: `/v3/users/1ff06b2d2aa94ede8619b6a895738e97` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:47.906583 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb9741-f731-46e0-86ad-6f9821957fba admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ff06b2d2aa94ede8619b6a895738e97)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:47.908488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb9741-f731-46e0-86ad-6f9821957fba admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:47.957243 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e1cb9741-f731-46e0-86ad-6f9821957fba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ff06b2d2aa94ede8619b6a895738e97'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:47.964636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e1cb9741-f731-46e0-86ad-6f9821957fba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ff06b2d2aa94ede8619b6a895738e97'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:47.976502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e1cb9741-f731-46e0-86ad-6f9821957fba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ff06b2d2aa94ede8619b6a895738e97'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:47.977424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e1cb9741-f731-46e0-86ad-6f9821957fba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ff06b2d2aa94ede8619b6a895738e97'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:47.978076 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e1cb9741-f731-46e0-86ad-6f9821957fba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ff06b2d2aa94ede8619b6a895738e97'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:47.992299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1148/2309] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:14:47 2022] DELETE /identity/v3/users/1ff06b2d2aa94ede8619b6a895738e97 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:47.997684 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5e546-0671-4300-996b-7aa44c08cd3e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:47.998148 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5e546-0671-4300-996b-7aa44c08cd3e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:47.998240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc5e546-0671-4300-996b-7aa44c08cd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:48.019408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-cfc5e546-0671-4300-996b-7aa44c08cd3e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:48.020613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cfc5e546-0671-4300-996b-7aa44c08cd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:48.052483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:48.082928 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1162/2310] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:48.093523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb9741-f731-46e0-86ad-6f9821957fba admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:48.094926 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e1cb9741-f731-46e0-86ad-6f9821957fba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:48.159463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb9741-f731-46e0-86ad-6f9821957fba admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:48.163929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e5738-a713-4f9a-8df6-488118372d50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:48.164562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e5738-a713-4f9a-8df6-488118372d50 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:48.164835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e5738-a713-4f9a-8df6-488118372d50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:48.165105 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e5738-a713-4f9a-8df6-488118372d50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:48.175505 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e5738-a713-4f9a-8df6-488118372d50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:48.176947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e5738-a713-4f9a-8df6-488118372d50 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:48.187457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1149/2311] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:14:48.576976 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5e546-0671-4300-996b-7aa44c08cd3e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:48.579094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cfc5e546-0671-4300-996b-7aa44c08cd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:48.601561 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc5e546-0671-4300-996b-7aa44c08cd3e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:48.606641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c90f1-f78b-45f4-be9d-2f377582eb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:48.607288 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c90f1-f78b-45f4-be9d-2f377582eb84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:48.607728 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c90f1-f78b-45f4-be9d-2f377582eb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:48.607816 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141c90f1-f78b-45f4-be9d-2f377582eb84 admin admin] PATH_INFO: `/v3/projects/024984cafe084754ac3544c7a40a1773` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:48.608944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c90f1-f78b-45f4-be9d-2f377582eb84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=024984cafe084754ac3544c7a40a1773)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:48.610974 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141c90f1-f78b-45f4-be9d-2f377582eb84 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:48.671665 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-141c90f1-f78b-45f4-be9d-2f377582eb84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '024984cafe084754ac3544c7a40a1773', 'request_id': 'req-141c90f1-f78b-45f4-be9d-2f377582eb84'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:48.677899 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-141c90f1-f78b-45f4-be9d-2f377582eb84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '024984cafe084754ac3544c7a40a1773', 'request_id': 'req-141c90f1-f78b-45f4-be9d-2f377582eb84'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:48.684074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-141c90f1-f78b-45f4-be9d-2f377582eb84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '024984cafe084754ac3544c7a40a1773', 'request_id': 'req-141c90f1-f78b-45f4-be9d-2f377582eb84'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:48.695506 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-141c90f1-f78b-45f4-be9d-2f377582eb84 admin admin] The token cache is being invalidate because project 024984cafe084754ac3544c7a40a1773 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:14:48.695794 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-141c90f1-f78b-45f4-be9d-2f377582eb84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:48.696408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-141c90f1-f78b-45f4-be9d-2f377582eb84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:48.697969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1163/2312] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:14:48 2022] DELETE /identity/v3/projects/024984cafe084754ac3544c7a40a1773 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:48.701931 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e5738-a713-4f9a-8df6-488118372d50 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:48.711416 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e5738-a713-4f9a-8df6-488118372d50 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:48.758114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:48.772053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c82e6f-ee9b-4e9a-bc7d-9b67d282f17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:48.772714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c82e6f-ee9b-4e9a-bc7d-9b67d282f17b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:48.772992 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c82e6f-ee9b-4e9a-bc7d-9b67d282f17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:48.773303 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c82e6f-ee9b-4e9a-bc7d-9b67d282f17b admin admin] PATH_INFO: `/v3/users/8f79b072567046da8531576e60cf5876` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:48.774409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c82e6f-ee9b-4e9a-bc7d-9b67d282f17b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f79b072567046da8531576e60cf5876)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:48.776773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c82e6f-ee9b-4e9a-bc7d-9b67d282f17b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:48.823479 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e7c82e6f-ee9b-4e9a-bc7d-9b67d282f17b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f79b072567046da8531576e60cf5876'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:48.830387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e7c82e6f-ee9b-4e9a-bc7d-9b67d282f17b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f79b072567046da8531576e60cf5876'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:48.842596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e7c82e6f-ee9b-4e9a-bc7d-9b67d282f17b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f79b072567046da8531576e60cf5876'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:48.843792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e7c82e6f-ee9b-4e9a-bc7d-9b67d282f17b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f79b072567046da8531576e60cf5876'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:48.844395 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e7c82e6f-ee9b-4e9a-bc7d-9b67d282f17b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f79b072567046da8531576e60cf5876'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:48.857451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1150/2313] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:14:48 2022] DELETE /identity/v3/users/8f79b072567046da8531576e60cf5876 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:49.130809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c90f1-f78b-45f4-be9d-2f377582eb84 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:49.132327 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-141c90f1-f78b-45f4-be9d-2f377582eb84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:49.147326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141c90f1-f78b-45f4-be9d-2f377582eb84 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:49.212478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0741c2b-d4da-45e7-87f8-12b4d441a65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:49.213723 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0741c2b-d4da-45e7-87f8-12b4d441a65e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:49.214008 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0741c2b-d4da-45e7-87f8-12b4d441a65e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:49.214283 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0741c2b-d4da-45e7-87f8-12b4d441a65e admin admin] PATH_INFO: `/v3/projects/4c2981493b4a40719c4f8aa3e55b577f` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:49.215427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0741c2b-d4da-45e7-87f8-12b4d441a65e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c2981493b4a40719c4f8aa3e55b577f)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:49.217369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0741c2b-d4da-45e7-87f8-12b4d441a65e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:49.268534 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a0741c2b-d4da-45e7-87f8-12b4d441a65e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c2981493b4a40719c4f8aa3e55b577f', 'request_id': 'req-a0741c2b-d4da-45e7-87f8-12b4d441a65e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:49.274988 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a0741c2b-d4da-45e7-87f8-12b4d441a65e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c2981493b4a40719c4f8aa3e55b577f', 'request_id': 'req-a0741c2b-d4da-45e7-87f8-12b4d441a65e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:49.281426 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a0741c2b-d4da-45e7-87f8-12b4d441a65e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c2981493b4a40719c4f8aa3e55b577f', 'request_id': 'req-a0741c2b-d4da-45e7-87f8-12b4d441a65e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:49.283107 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:49.296754 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a0741c2b-d4da-45e7-87f8-12b4d441a65e admin admin] The token cache is being invalidate because project 4c2981493b4a40719c4f8aa3e55b577f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:14:49.297045 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a0741c2b-d4da-45e7-87f8-12b4d441a65e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:49.297711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a0741c2b-d4da-45e7-87f8-12b4d441a65e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:49.299210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1164/2314] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:14:49 2022] DELETE /identity/v3/projects/4c2981493b4a40719c4f8aa3e55b577f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:49.304332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf42e6-e90f-460d-a661-a1bb72bb2ee0 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:49.304636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf42e6-e90f-460d-a661-a1bb72bb2ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:49.304907 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf42e6-e90f-460d-a661-a1bb72bb2ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:49.325709 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-95cf42e6-e90f-460d-a661-a1bb72bb2ee0 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:49.328982 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-95cf42e6-e90f-460d-a661-a1bb72bb2ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:49.403716 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1151/2315] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:14:49.413868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee8316-d612-4947-a74a-8ffbde4eb332 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:49.413868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee8316-d612-4947-a74a-8ffbde4eb332 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:49.413868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee8316-d612-4947-a74a-8ffbde4eb332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:49.438078 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-63ee8316-d612-4947-a74a-8ffbde4eb332 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:49.439280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-63ee8316-d612-4947-a74a-8ffbde4eb332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:49.447880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1165/2316] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:49.452337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf42e6-e90f-460d-a661-a1bb72bb2ee0 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:49.465320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf42e6-e90f-460d-a661-a1bb72bb2ee0 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:49.470028 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7695a-14e6-44ee-ad0a-8c46a8264716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:49.470708 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7695a-14e6-44ee-ad0a-8c46a8264716 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:49.470981 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7695a-14e6-44ee-ad0a-8c46a8264716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:49.471249 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d7695a-14e6-44ee-ad0a-8c46a8264716 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:49.471901 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7695a-14e6-44ee-ad0a-8c46a8264716 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:14:49.472204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7695a-14e6-44ee-ad0a-8c46a8264716 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:49.474483 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d7695a-14e6-44ee-ad0a-8c46a8264716 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:49.487831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1152/2317] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:14:49.492509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee8316-d612-4947-a74a-8ffbde4eb332 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:49.503043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee8316-d612-4947-a74a-8ffbde4eb332 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:49.510453 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75be2c1-d8d4-4a68-89c8-47735b18a80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:49.511051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75be2c1-d8d4-4a68-89c8-47735b18a80f admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:49.511398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75be2c1-d8d4-4a68-89c8-47735b18a80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:49.511733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75be2c1-d8d4-4a68-89c8-47735b18a80f admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:49.512422 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75be2c1-d8d4-4a68-89c8-47735b18a80f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:49.514616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75be2c1-d8d4-4a68-89c8-47735b18a80f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:49.551956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1166/2318] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:14:49 2022] POST /identity/v3/projects => generated 377 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:49.556269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7695a-14e6-44ee-ad0a-8c46a8264716 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:49.566857 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d7695a-14e6-44ee-ad0a-8c46a8264716 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:49.620961 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:49.629753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8add4b6-0992-49fe-a725-2f87a866366e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:49.630522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8add4b6-0992-49fe-a725-2f87a866366e admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:49.631002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8add4b6-0992-49fe-a725-2f87a866366e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:49.631346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8add4b6-0992-49fe-a725-2f87a866366e admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:49.632071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8add4b6-0992-49fe-a725-2f87a866366e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:49.637013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8add4b6-0992-49fe-a725-2f87a866366e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:49.679840 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1153/2319] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:14:49 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:14:49.681053 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75be2c1-d8d4-4a68-89c8-47735b18a80f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:49.694097 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75be2c1-d8d4-4a68-89c8-47735b18a80f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:49.700534 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b13f17-fb37-4e48-b885-8e60b4772ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:49.701328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b13f17-fb37-4e48-b885-8e60b4772ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:49.701509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b13f17-fb37-4e48-b885-8e60b4772ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:49.701810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b13f17-fb37-4e48-b885-8e60b4772ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:49.702524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b13f17-fb37-4e48-b885-8e60b4772ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:49.704107 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b13f17-fb37-4e48-b885-8e60b4772ec7 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:49.717949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1167/2320] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:14:49.719730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8add4b6-0992-49fe-a725-2f87a866366e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:49.731494 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8add4b6-0992-49fe-a725-2f87a866366e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:49.736338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2071b69-0014-4c37-9e75-c738bc953fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:49.736964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2071b69-0014-4c37-9e75-c738bc953fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:49.737277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2071b69-0014-4c37-9e75-c738bc953fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:49.737553 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2071b69-0014-4c37-9e75-c738bc953fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:49.738193 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2071b69-0014-4c37-9e75-c738bc953fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:49.739775 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2071b69-0014-4c37-9e75-c738bc953fe6 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:49.756937 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1154/2321] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:14:49.761307 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b13f17-fb37-4e48-b885-8e60b4772ec7 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:49.781949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b13f17-fb37-4e48-b885-8e60b4772ec7 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:49.790486 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15b4712-9642-4c93-aa8b-06cbd9860c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:49.791189 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b4712-9642-4c93-aa8b-06cbd9860c33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:49.791465 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b4712-9642-4c93-aa8b-06cbd9860c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:49.791925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15b4712-9642-4c93-aa8b-06cbd9860c33 admin admin] PATH_INFO: `/v3/projects/e77fcc563adb43d9b38c5a944969bf2d/users/0da78d6f291c438baa9d1fa47ded4c8e/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:49.798877 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2071b69-0014-4c37-9e75-c738bc953fe6 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:49.800911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a2071b69-0014-4c37-9e75-c738bc953fe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:49.812805 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b4712-9642-4c93-aa8b-06cbd9860c33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e77fcc563adb43d9b38c5a944969bf2d, user_id=0da78d6f291c438baa9d1fa47ded4c8e, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:49.815657 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15b4712-9642-4c93-aa8b-06cbd9860c33 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:49.816459 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2071b69-0014-4c37-9e75-c738bc953fe6 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:49.830259 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28e4d04-964f-4636-a7b9-b6eb5322440a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:49.831200 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e4d04-964f-4636-a7b9-b6eb5322440a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:49.831764 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e4d04-964f-4636-a7b9-b6eb5322440a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:49.831955 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28e4d04-964f-4636-a7b9-b6eb5322440a admin admin] PATH_INFO: `/v3/users/842d761d9eba453ba006477f5b8c58db` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:49.833685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e4d04-964f-4636-a7b9-b6eb5322440a admin admin] RBAC: Authorizing `identity:delete_user(user_id=842d761d9eba453ba006477f5b8c58db)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:49.837217 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1168/2322] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:49 2022] PUT /identity/v3/projects/e77fcc563adb43d9b38c5a944969bf2d/users/0da78d6f291c438baa9d1fa47ded4c8e/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:49.837369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28e4d04-964f-4636-a7b9-b6eb5322440a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:49.845862 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f395f4b-4559-4326-9e77-c290ddeefdba None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:49.846248 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f395f4b-4559-4326-9e77-c290ddeefdba None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:49.846522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f395f4b-4559-4326-9e77-c290ddeefdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:49.899318 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7f395f4b-4559-4326-9e77-c290ddeefdba None None] MFA Rules not processed for user `0da78d6f291c438baa9d1fa47ded4c8e`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:49.900957 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7f395f4b-4559-4326-9e77-c290ddeefdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:49.907666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a28e4d04-964f-4636-a7b9-b6eb5322440a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '842d761d9eba453ba006477f5b8c58db'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:49.917529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a28e4d04-964f-4636-a7b9-b6eb5322440a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '842d761d9eba453ba006477f5b8c58db'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:49.944800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a28e4d04-964f-4636-a7b9-b6eb5322440a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '842d761d9eba453ba006477f5b8c58db'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:49.945761 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a28e4d04-964f-4636-a7b9-b6eb5322440a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '842d761d9eba453ba006477f5b8c58db'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:49.946685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a28e4d04-964f-4636-a7b9-b6eb5322440a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '842d761d9eba453ba006477f5b8c58db'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:49.978716 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1155/2323] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:14:49 2022] DELETE /identity/v3/users/842d761d9eba453ba006477f5b8c58db => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:49.988317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03865a4-87f1-4fe4-9fd0-14162b6af779 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:49.988629 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03865a4-87f1-4fe4-9fd0-14162b6af779 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:49.988898 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03865a4-87f1-4fe4-9fd0-14162b6af779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:50.000040 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1169/2324] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:50.011936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e03865a4-87f1-4fe4-9fd0-14162b6af779 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:50.013151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e03865a4-87f1-4fe4-9fd0-14162b6af779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:50.071919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1156/2325] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:50.082115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f395f4b-4559-4326-9e77-c290ddeefdba None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:50.083625 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7f395f4b-4559-4326-9e77-c290ddeefdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:50.140204 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f395f4b-4559-4326-9e77-c290ddeefdba None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:50.169584 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:50.190992 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf54d16e-a660-41c0-8fcf-2ce079d4efb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:50.191746 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54d16e-a660-41c0-8fcf-2ce079d4efb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:50.192022 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54d16e-a660-41c0-8fcf-2ce079d4efb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:50.192292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54d16e-a660-41c0-8fcf-2ce079d4efb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:50.201516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf54d16e-a660-41c0-8fcf-2ce079d4efb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:50.202928 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf54d16e-a660-41c0-8fcf-2ce079d4efb7 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:50.212403 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1170/2326] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:14:50.498513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03865a4-87f1-4fe4-9fd0-14162b6af779 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:50.499969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e03865a4-87f1-4fe4-9fd0-14162b6af779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:50.515567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03865a4-87f1-4fe4-9fd0-14162b6af779 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:50.520312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:50.520949 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:50.521248 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:50.521524 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7 admin admin] PATH_INFO: `/v3/projects/1fb9cca799f94645b7fbb55e1f8fc8ad` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:50.522646 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fb9cca799f94645b7fbb55e1f8fc8ad)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:50.524537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:50.569932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=83,keystone:INSERT=4,keystone:DELETE=13 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:50.577827 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fb9cca799f94645b7fbb55e1f8fc8ad', 'request_id': 'req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:50.587300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fb9cca799f94645b7fbb55e1f8fc8ad', 'request_id': 'req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:50.599015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fb9cca799f94645b7fbb55e1f8fc8ad', 'request_id': 'req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:50.613673 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7 admin admin] The token cache is being invalidate because project 1fb9cca799f94645b7fbb55e1f8fc8ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:14:50.613871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:50.614518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:50.616084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1157/2327] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:14:50 2022] DELETE /identity/v3/projects/1fb9cca799f94645b7fbb55e1f8fc8ad => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:50.622311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf54d16e-a660-41c0-8fcf-2ce079d4efb7 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:50.632693 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf54d16e-a660-41c0-8fcf-2ce079d4efb7 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:50.700968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00aa55-523c-431f-95f2-5375b737afd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:50.701652 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00aa55-523c-431f-95f2-5375b737afd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:50.701951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00aa55-523c-431f-95f2-5375b737afd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:50.702207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b00aa55-523c-431f-95f2-5375b737afd9 admin admin] PATH_INFO: `/v3/users/6975c49fc5dc4b2ab388c89a4cb93ae2` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:50.703363 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00aa55-523c-431f-95f2-5375b737afd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6975c49fc5dc4b2ab388c89a4cb93ae2)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:50.705329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b00aa55-523c-431f-95f2-5375b737afd9 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:50.756354 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6b00aa55-523c-431f-95f2-5375b737afd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6975c49fc5dc4b2ab388c89a4cb93ae2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:50.763300 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6b00aa55-523c-431f-95f2-5375b737afd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6975c49fc5dc4b2ab388c89a4cb93ae2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:50.770924 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6b00aa55-523c-431f-95f2-5375b737afd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6975c49fc5dc4b2ab388c89a4cb93ae2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:50.773602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6b00aa55-523c-431f-95f2-5375b737afd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6975c49fc5dc4b2ab388c89a4cb93ae2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:50.787958 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-6b00aa55-523c-431f-95f2-5375b737afd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6975c49fc5dc4b2ab388c89a4cb93ae2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:50.794515 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1171/2328] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:14:50 2022] DELETE /identity/v3/users/6975c49fc5dc4b2ab388c89a4cb93ae2 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:51.297853 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:51.300042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:51.314915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca8a0a2-a056-4ef5-8432-2f10d9023dd7 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:51.374873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd372e-75e5-488c-9a44-62a766498899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:51.375692 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd372e-75e5-488c-9a44-62a766498899 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:51.376030 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd372e-75e5-488c-9a44-62a766498899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:51.376359 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abd372e-75e5-488c-9a44-62a766498899 admin admin] PATH_INFO: `/v3/projects/c3a6f580e64f40178a4d7bd571e4d516` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:51.377590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd372e-75e5-488c-9a44-62a766498899 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3a6f580e64f40178a4d7bd571e4d516)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:51.379502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abd372e-75e5-488c-9a44-62a766498899 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:51.429472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6abd372e-75e5-488c-9a44-62a766498899 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3a6f580e64f40178a4d7bd571e4d516', 'request_id': 'req-6abd372e-75e5-488c-9a44-62a766498899'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:51.437350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6abd372e-75e5-488c-9a44-62a766498899 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3a6f580e64f40178a4d7bd571e4d516', 'request_id': 'req-6abd372e-75e5-488c-9a44-62a766498899'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:51.443409 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6abd372e-75e5-488c-9a44-62a766498899 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3a6f580e64f40178a4d7bd571e4d516', 'request_id': 'req-6abd372e-75e5-488c-9a44-62a766498899'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:51.457942 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6abd372e-75e5-488c-9a44-62a766498899 admin admin] The token cache is being invalidate because project c3a6f580e64f40178a4d7bd571e4d516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:14:51.458225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6abd372e-75e5-488c-9a44-62a766498899 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:51.458871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-6abd372e-75e5-488c-9a44-62a766498899 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:51.460355 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1158/2329] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:14:51 2022] DELETE /identity/v3/projects/c3a6f580e64f40178a4d7bd571e4d516 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:51.464581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00aa55-523c-431f-95f2-5375b737afd9 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:51.465777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6b00aa55-523c-431f-95f2-5375b737afd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:51.479024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b00aa55-523c-431f-95f2-5375b737afd9 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:51.533898 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:51.542126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab2762f-82f8-46f5-b9f8-824850743f40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:51.542952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2762f-82f8-46f5-b9f8-824850743f40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:51.543411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2762f-82f8-46f5-b9f8-824850743f40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:51.543683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab2762f-82f8-46f5-b9f8-824850743f40 admin admin] PATH_INFO: `/v3/users/b107cf83f7ac4b91a63c665ff86890ee` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:51.544882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2762f-82f8-46f5-b9f8-824850743f40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b107cf83f7ac4b91a63c665ff86890ee)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:51.546989 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab2762f-82f8-46f5-b9f8-824850743f40 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:51.594901 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7ab2762f-82f8-46f5-b9f8-824850743f40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b107cf83f7ac4b91a63c665ff86890ee'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:51.601709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7ab2762f-82f8-46f5-b9f8-824850743f40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b107cf83f7ac4b91a63c665ff86890ee'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:51.610817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7ab2762f-82f8-46f5-b9f8-824850743f40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b107cf83f7ac4b91a63c665ff86890ee'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:51.611366 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7ab2762f-82f8-46f5-b9f8-824850743f40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b107cf83f7ac4b91a63c665ff86890ee'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:51.623854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7ab2762f-82f8-46f5-b9f8-824850743f40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b107cf83f7ac4b91a63c665ff86890ee'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:51.630446 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1172/2330] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:14:51 2022] DELETE /identity/v3/users/b107cf83f7ac4b91a63c665ff86890ee => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:14:51.983525 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd372e-75e5-488c-9a44-62a766498899 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:51.984939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6abd372e-75e5-488c-9a44-62a766498899 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:52.001087 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abd372e-75e5-488c-9a44-62a766498899 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:52.063443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852bf75d-2792-45ca-b222-9180463ee98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:52.064074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bf75d-2792-45ca-b222-9180463ee98e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:52.064351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bf75d-2792-45ca-b222-9180463ee98e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:52.064624 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852bf75d-2792-45ca-b222-9180463ee98e admin admin] PATH_INFO: `/v3/projects/df60022cfb934d5095ee645fd498006f` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:52.065781 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852bf75d-2792-45ca-b222-9180463ee98e admin admin] RBAC: Authorizing `identity:delete_project(project_id=df60022cfb934d5095ee645fd498006f)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:52.067861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852bf75d-2792-45ca-b222-9180463ee98e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:52.069522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:52.123705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-852bf75d-2792-45ca-b222-9180463ee98e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df60022cfb934d5095ee645fd498006f', 'request_id': 'req-852bf75d-2792-45ca-b222-9180463ee98e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:52.133928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-852bf75d-2792-45ca-b222-9180463ee98e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df60022cfb934d5095ee645fd498006f', 'request_id': 'req-852bf75d-2792-45ca-b222-9180463ee98e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:52.140177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-852bf75d-2792-45ca-b222-9180463ee98e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df60022cfb934d5095ee645fd498006f', 'request_id': 'req-852bf75d-2792-45ca-b222-9180463ee98e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:52.153486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-852bf75d-2792-45ca-b222-9180463ee98e admin admin] The token cache is being invalidate because project df60022cfb934d5095ee645fd498006f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:14:52.153486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-852bf75d-2792-45ca-b222-9180463ee98e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:52.154141 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-852bf75d-2792-45ca-b222-9180463ee98e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:52.156450 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1159/2331] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:14:51 2022] DELETE /identity/v3/projects/df60022cfb934d5095ee645fd498006f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:52.161490 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511a8a8-05bc-4d7f-b073-315655d10a9f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:52.161823 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511a8a8-05bc-4d7f-b073-315655d10a9f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:52.162122 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511a8a8-05bc-4d7f-b073-315655d10a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:52.182266 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f511a8a8-05bc-4d7f-b073-315655d10a9f None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:52.183457 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f511a8a8-05bc-4d7f-b073-315655d10a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:52.254746 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1173/2332] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:52.263370 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ee38d-3f70-4baf-a5fb-7fd20a6bad4a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:52.263762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ee38d-3f70-4baf-a5fb-7fd20a6bad4a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:52.265621 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ee38d-3f70-4baf-a5fb-7fd20a6bad4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:52.291613 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b86ee38d-3f70-4baf-a5fb-7fd20a6bad4a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:52.293368 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b86ee38d-3f70-4baf-a5fb-7fd20a6bad4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:52.303981 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1160/2333] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:52.308999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511a8a8-05bc-4d7f-b073-315655d10a9f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:52.325586 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511a8a8-05bc-4d7f-b073-315655d10a9f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:52.331404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f774c8-ef26-45ea-9ec2-e1bcad75511e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:52.331602 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f774c8-ef26-45ea-9ec2-e1bcad75511e admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:52.331758 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f774c8-ef26-45ea-9ec2-e1bcad75511e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:52.332025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f774c8-ef26-45ea-9ec2-e1bcad75511e admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:52.332675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f774c8-ef26-45ea-9ec2-e1bcad75511e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:14:52.332675 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f774c8-ef26-45ea-9ec2-e1bcad75511e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:52.334681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f774c8-ef26-45ea-9ec2-e1bcad75511e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:52.355780 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1174/2334] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:14:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:14:52.357350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ee38d-3f70-4baf-a5fb-7fd20a6bad4a None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:52.369764 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ee38d-3f70-4baf-a5fb-7fd20a6bad4a None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:52.375726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f470d7-0a8b-414d-9d81-09e5d7b249eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:52.376540 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f470d7-0a8b-414d-9d81-09e5d7b249eb admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:52.376935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f470d7-0a8b-414d-9d81-09e5d7b249eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:52.377282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f470d7-0a8b-414d-9d81-09e5d7b249eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:52.378064 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f470d7-0a8b-414d-9d81-09e5d7b249eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:52.380323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f470d7-0a8b-414d-9d81-09e5d7b249eb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:52.413147 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1161/2335] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:14:52 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:14:52.422803 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f774c8-ef26-45ea-9ec2-e1bcad75511e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:52.437264 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f774c8-ef26-45ea-9ec2-e1bcad75511e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:52.515649 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636f359-4a52-45ba-900e-a814e1beb48f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:52.516275 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636f359-4a52-45ba-900e-a814e1beb48f admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:52.516510 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636f359-4a52-45ba-900e-a814e1beb48f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:52.516944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636f359-4a52-45ba-900e-a814e1beb48f admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:52.517763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636f359-4a52-45ba-900e-a814e1beb48f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:52.519828 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636f359-4a52-45ba-900e-a814e1beb48f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:52.533955 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:52.548664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1175/2336] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:14:52 2022] POST /identity/v3/users => generated 287 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:14:52.552895 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f470d7-0a8b-414d-9d81-09e5d7b249eb admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:52.565248 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f470d7-0a8b-414d-9d81-09e5d7b249eb admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:52.570008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60ff59-34fe-405d-8767-2cad26025e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:52.570606 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60ff59-34fe-405d-8767-2cad26025e4f admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:52.570881 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60ff59-34fe-405d-8767-2cad26025e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:52.571142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a60ff59-34fe-405d-8767-2cad26025e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:52.571771 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60ff59-34fe-405d-8767-2cad26025e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:52.573929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a60ff59-34fe-405d-8767-2cad26025e4f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:52.584346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1162/2337] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:14:52.589365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636f359-4a52-45ba-900e-a814e1beb48f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:52.599496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636f359-4a52-45ba-900e-a814e1beb48f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:52.604341 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9a48a4-022d-41a3-826b-e83c0092733b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:52.604943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a48a4-022d-41a3-826b-e83c0092733b admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:52.605270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a48a4-022d-41a3-826b-e83c0092733b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:52.605549 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9a48a4-022d-41a3-826b-e83c0092733b admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:52.606461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a48a4-022d-41a3-826b-e83c0092733b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:52.608035 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9a48a4-022d-41a3-826b-e83c0092733b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:52.618581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1176/2338] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:14:52.622739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60ff59-34fe-405d-8767-2cad26025e4f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:52.633391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a60ff59-34fe-405d-8767-2cad26025e4f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:52.638591 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d88851-8411-4834-98f4-6fce9e0c131d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:52.639227 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d88851-8411-4834-98f4-6fce9e0c131d admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:52.639501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d88851-8411-4834-98f4-6fce9e0c131d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:52.639770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d88851-8411-4834-98f4-6fce9e0c131d admin admin] PATH_INFO: `/v3/projects/f5e4dacbe64a48f49f697787961e1200/users/426bb2c78bce40179a0a115651b81a5b/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:52.654851 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d88851-8411-4834-98f4-6fce9e0c131d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5e4dacbe64a48f49f697787961e1200, user_id=426bb2c78bce40179a0a115651b81a5b, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:52.657569 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d88851-8411-4834-98f4-6fce9e0c131d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:52.670281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1163/2339] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:14:52 2022] PUT /identity/v3/projects/f5e4dacbe64a48f49f697787961e1200/users/426bb2c78bce40179a0a115651b81a5b/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:52.680449 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e572b3-5d06-45c9-be07-55d50bae2a1a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:52.680735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e572b3-5d06-45c9-be07-55d50bae2a1a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:52.681010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e572b3-5d06-45c9-be07-55d50bae2a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:52.708852 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-61e572b3-5d06-45c9-be07-55d50bae2a1a None None] MFA Rules not processed for user `426bb2c78bce40179a0a115651b81a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:52.710087 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-61e572b3-5d06-45c9-be07-55d50bae2a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:52.773375 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1177/2340] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:52.784670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d88851-8411-4834-98f4-6fce9e0c131d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:52.786373 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d2d88851-8411-4834-98f4-6fce9e0c131d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:52.844844 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d88851-8411-4834-98f4-6fce9e0c131d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:52.849011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b359d-d02d-44b6-9f6b-8a34afc55e5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:52.849675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b359d-d02d-44b6-9f6b-8a34afc55e5c service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:52.849952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b359d-d02d-44b6-9f6b-8a34afc55e5c service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:52.850220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b359d-d02d-44b6-9f6b-8a34afc55e5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:52.859787 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b359d-d02d-44b6-9f6b-8a34afc55e5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:52.861261 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b359d-d02d-44b6-9f6b-8a34afc55e5c service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:52.870841 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1164/2341] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:14:53.715630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e572b3-5d06-45c9-be07-55d50bae2a1a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:53.717126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-61e572b3-5d06-45c9-be07-55d50bae2a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:53.775567 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e572b3-5d06-45c9-be07-55d50bae2a1a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:53.784058 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d02c47-77de-4751-8bca-374885fdaf05 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1863336fecad4987acdfd0b3f7b2f70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:53.784694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d02c47-77de-4751-8bca-374885fdaf05 service glance] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:53.784979 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d02c47-77de-4751-8bca-374885fdaf05 service glance] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:53.785284 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d02c47-77de-4751-8bca-374885fdaf05 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:53.787200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e5d02c47-77de-4751-8bca-374885fdaf05 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:53.847330 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d02c47-77de-4751-8bca-374885fdaf05 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:53.848756 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d02c47-77de-4751-8bca-374885fdaf05 service glance] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:53.858489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1178/2342] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:14:55.334825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04a65b-6d04-4706-9c6e-ea094ae51733 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:55.335366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04a65b-6d04-4706-9c6e-ea094ae51733 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:55.335366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04a65b-6d04-4706-9c6e-ea094ae51733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:55.356704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ca04a65b-6d04-4706-9c6e-ea094ae51733 None None] MFA Rules not processed for user `426bb2c78bce40179a0a115651b81a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:55.358028 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ca04a65b-6d04-4706-9c6e-ea094ae51733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:55.365503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1165/2343] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:58.986450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d02c47-77de-4751-8bca-374885fdaf05 service glance] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:58.987855 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e5d02c47-77de-4751-8bca-374885fdaf05 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:59.022093 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:59.065480 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d02c47-77de-4751-8bca-374885fdaf05 service glance] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:59.071587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddd6ac-1cc3-4f2d-b7ad-4afed4e02c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:59.072398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddd6ac-1cc3-4f2d-b7ad-4afed4e02c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:59.072681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddd6ac-1cc3-4f2d-b7ad-4afed4e02c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:59.073626 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ddd6ac-1cc3-4f2d-b7ad-4afed4e02c9d admin admin] PATH_INFO: `/v3/users/19774677040b44a0adff88b6563b8ff9` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:59.074848 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ddd6ac-1cc3-4f2d-b7ad-4afed4e02c9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=19774677040b44a0adff88b6563b8ff9)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:59.076787 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ddd6ac-1cc3-4f2d-b7ad-4afed4e02c9d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:59.126592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e7ddd6ac-1cc3-4f2d-b7ad-4afed4e02c9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19774677040b44a0adff88b6563b8ff9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:59.141279 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e7ddd6ac-1cc3-4f2d-b7ad-4afed4e02c9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19774677040b44a0adff88b6563b8ff9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:59.150301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e7ddd6ac-1cc3-4f2d-b7ad-4afed4e02c9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19774677040b44a0adff88b6563b8ff9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:59.150920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e7ddd6ac-1cc3-4f2d-b7ad-4afed4e02c9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19774677040b44a0adff88b6563b8ff9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:59.165093 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e7ddd6ac-1cc3-4f2d-b7ad-4afed4e02c9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19774677040b44a0adff88b6563b8ff9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:59.171882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1179/2344] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:14:58 2022] DELETE /identity/v3/users/19774677040b44a0adff88b6563b8ff9 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:59.181760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f0195-4cc6-4ab7-a2f0-11c501874e5c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:59.182336 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f0195-4cc6-4ab7-a2f0-11c501874e5c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:59.182336 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f0195-4cc6-4ab7-a2f0-11c501874e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:59.206147 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-980f0195-4cc6-4ab7-a2f0-11c501874e5c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:59.207738 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-980f0195-4cc6-4ab7-a2f0-11c501874e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:59.237355 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:59.306180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1166/2345] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:59.319938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddd6ac-1cc3-4f2d-b7ad-4afed4e02c9d admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:59.321832 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e7ddd6ac-1cc3-4f2d-b7ad-4afed4e02c9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:59.385918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ddd6ac-1cc3-4f2d-b7ad-4afed4e02c9d admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:59.390510 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcdb974-143c-4d5f-8a72-7e0f28c0575d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:59.391222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdb974-143c-4d5f-8a72-7e0f28c0575d service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:59.391571 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdb974-143c-4d5f-8a72-7e0f28c0575d service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:59.391923 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcdb974-143c-4d5f-8a72-7e0f28c0575d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:59.401231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcdb974-143c-4d5f-8a72-7e0f28c0575d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:59.402733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcdb974-143c-4d5f-8a72-7e0f28c0575d service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:59.412792 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1180/2346] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:14:59.701595 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f0195-4cc6-4ab7-a2f0-11c501874e5c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:59.703585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-980f0195-4cc6-4ab7-a2f0-11c501874e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:59.718851 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f0195-4cc6-4ab7-a2f0-11c501874e5c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:59.723570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcf165-c883-4d36-8553-0e7b80def66b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:59.724279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcf165-c883-4d36-8553-0e7b80def66b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:59.724661 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcf165-c883-4d36-8553-0e7b80def66b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:59.725045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcf165-c883-4d36-8553-0e7b80def66b admin admin] PATH_INFO: `/v3/projects/a2ed52b48e6f4c28880ebe8e3b0cb17c` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:59.726267 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebcf165-c883-4d36-8553-0e7b80def66b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2ed52b48e6f4c28880ebe8e3b0cb17c)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:59.729016 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebcf165-c883-4d36-8553-0e7b80def66b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:59.782332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2ebcf165-c883-4d36-8553-0e7b80def66b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2ed52b48e6f4c28880ebe8e3b0cb17c', 'request_id': 'req-2ebcf165-c883-4d36-8553-0e7b80def66b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:59.791331 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2ebcf165-c883-4d36-8553-0e7b80def66b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2ed52b48e6f4c28880ebe8e3b0cb17c', 'request_id': 'req-2ebcf165-c883-4d36-8553-0e7b80def66b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:59.797114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2ebcf165-c883-4d36-8553-0e7b80def66b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2ed52b48e6f4c28880ebe8e3b0cb17c', 'request_id': 'req-2ebcf165-c883-4d36-8553-0e7b80def66b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:59.809133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2ebcf165-c883-4d36-8553-0e7b80def66b admin admin] The token cache is being invalidate because project a2ed52b48e6f4c28880ebe8e3b0cb17c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:14:59.809450 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2ebcf165-c883-4d36-8553-0e7b80def66b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:59.810185 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2ebcf165-c883-4d36-8553-0e7b80def66b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:14:59.811746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1167/2347] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:14:59 2022] DELETE /identity/v3/projects/a2ed52b48e6f4c28880ebe8e3b0cb17c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:14:59.816724 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29e99f-d1ae-459b-abd9-38e30fe12372 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:59.817027 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29e99f-d1ae-459b-abd9-38e30fe12372 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:59.817337 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29e99f-d1ae-459b-abd9-38e30fe12372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:59.838990 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-6f29e99f-d1ae-459b-abd9-38e30fe12372 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:59.840183 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6f29e99f-d1ae-459b-abd9-38e30fe12372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:59.882822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:14:59.902443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1181/2348] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:59.910411 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f8aed-0ccd-42fd-83cf-d164e4fba93a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:59.910676 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f8aed-0ccd-42fd-83cf-d164e4fba93a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:59.910942 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f8aed-0ccd-42fd-83cf-d164e4fba93a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:59.931814 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4a5f8aed-0ccd-42fd-83cf-d164e4fba93a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:14:59.933024 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4a5f8aed-0ccd-42fd-83cf-d164e4fba93a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:14:59.945300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1168/2349] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:14:59.946605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29e99f-d1ae-459b-abd9-38e30fe12372 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:59.957348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29e99f-d1ae-459b-abd9-38e30fe12372 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:59.962044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6222d-20e2-4908-888c-5c2676cb91d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:59.973725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6222d-20e2-4908-888c-5c2676cb91d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:59.973725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6222d-20e2-4908-888c-5c2676cb91d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:59.973725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6222d-20e2-4908-888c-5c2676cb91d0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:59.973725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6222d-20e2-4908-888c-5c2676cb91d0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:14:59.973725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6222d-20e2-4908-888c-5c2676cb91d0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:14:59.973725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6222d-20e2-4908-888c-5c2676cb91d0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:14:59.978094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1182/2350] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:14:59.982657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f8aed-0ccd-42fd-83cf-d164e4fba93a None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:14:59.992137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f8aed-0ccd-42fd-83cf-d164e4fba93a None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:14:59.996862 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a7e09-cb84-49f8-b342-7eea1372c827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:14:59.997490 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a7e09-cb84-49f8-b342-7eea1372c827 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:14:59.997763 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a7e09-cb84-49f8-b342-7eea1372c827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:14:59.998029 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31a7e09-cb84-49f8-b342-7eea1372c827 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:14:59.998686 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a7e09-cb84-49f8-b342-7eea1372c827 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:00.000566 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31a7e09-cb84-49f8-b342-7eea1372c827 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:00.030141 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1169/2351] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:14:59 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:15:00.034519 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6222d-20e2-4908-888c-5c2676cb91d0 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:00.044668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6222d-20e2-4908-888c-5c2676cb91d0 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:00.102131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ee5e1-8498-4785-83a1-ea51eb9f323d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:00.102741 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ee5e1-8498-4785-83a1-ea51eb9f323d admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:00.103055 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ee5e1-8498-4785-83a1-ea51eb9f323d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:00.103328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82ee5e1-8498-4785-83a1-ea51eb9f323d admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:00.103971 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ee5e1-8498-4785-83a1-ea51eb9f323d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:00.106185 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82ee5e1-8498-4785-83a1-ea51eb9f323d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:00.126697 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1183/2352] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:15:00 2022] POST /identity/v3/users => generated 302 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:15:00.130963 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a7e09-cb84-49f8-b342-7eea1372c827 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:00.142198 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31a7e09-cb84-49f8-b342-7eea1372c827 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:00.146830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176fa0a-a866-4834-9117-d7179936bf16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:00.147428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176fa0a-a866-4834-9117-d7179936bf16 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:00.147699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176fa0a-a866-4834-9117-d7179936bf16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:00.147965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f176fa0a-a866-4834-9117-d7179936bf16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:00.148597 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176fa0a-a866-4834-9117-d7179936bf16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:00.150784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f176fa0a-a866-4834-9117-d7179936bf16 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:00.160959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1170/2353] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:15:00.165122 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ee5e1-8498-4785-83a1-ea51eb9f323d admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:00.175429 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82ee5e1-8498-4785-83a1-ea51eb9f323d admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:00.180040 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4d5f15-404a-453d-a702-818f82e29d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:00.180638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d5f15-404a-453d-a702-818f82e29d33 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:00.180916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d5f15-404a-453d-a702-818f82e29d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:00.181209 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4d5f15-404a-453d-a702-818f82e29d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:00.181850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d5f15-404a-453d-a702-818f82e29d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:00.183417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4d5f15-404a-453d-a702-818f82e29d33 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:00.194518 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1184/2354] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:15:00.201883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176fa0a-a866-4834-9117-d7179936bf16 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:00.212231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f176fa0a-a866-4834-9117-d7179936bf16 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:00.217772 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95549cdb-fbdc-4fc0-b329-60d7a54ca000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:00.218431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95549cdb-fbdc-4fc0-b329-60d7a54ca000 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:00.218722 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95549cdb-fbdc-4fc0-b329-60d7a54ca000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:00.218984 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95549cdb-fbdc-4fc0-b329-60d7a54ca000 admin admin] PATH_INFO: `/v3/projects/61a4fb2c4c654a9db5c64b9d7071851a/users/57d53facfca6424386b35a42255e8baf/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:00.236119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95549cdb-fbdc-4fc0-b329-60d7a54ca000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a4fb2c4c654a9db5c64b9d7071851a, user_id=57d53facfca6424386b35a42255e8baf, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:00.238839 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95549cdb-fbdc-4fc0-b329-60d7a54ca000 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:00.256479 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1171/2355] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:15:00 2022] PUT /identity/v3/projects/61a4fb2c4c654a9db5c64b9d7071851a/users/57d53facfca6424386b35a42255e8baf/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:00.258574 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046232e5-7ae1-40d3-a490-71fcbabc8c6f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:00.258926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046232e5-7ae1-40d3-a490-71fcbabc8c6f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:00.259852 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046232e5-7ae1-40d3-a490-71fcbabc8c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:00.279316 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-046232e5-7ae1-40d3-a490-71fcbabc8c6f None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:00.280619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-046232e5-7ae1-40d3-a490-71fcbabc8c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:00.360676 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1185/2356] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:00.371261 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95549cdb-fbdc-4fc0-b329-60d7a54ca000 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:00.373089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-95549cdb-fbdc-4fc0-b329-60d7a54ca000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:00.437338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:00.454226 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95549cdb-fbdc-4fc0-b329-60d7a54ca000 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:00.459260 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df96e5e-913a-4769-ad90-8d66b0b8530d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:00.459945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df96e5e-913a-4769-ad90-8d66b0b8530d service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:00.460298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df96e5e-913a-4769-ad90-8d66b0b8530d service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:00.460636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df96e5e-913a-4769-ad90-8d66b0b8530d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:00.478351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df96e5e-913a-4769-ad90-8d66b0b8530d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:00.480686 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df96e5e-913a-4769-ad90-8d66b0b8530d service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:00.494993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1172/2357] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:01.437823 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de7b6f-ee4f-40fd-9df3-c3ec7a4a33dd None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:01.438315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de7b6f-ee4f-40fd-9df3-c3ec7a4a33dd None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:01.438605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69de7b6f-ee4f-40fd-9df3-c3ec7a4a33dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:01.476632 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-69de7b6f-ee4f-40fd-9df3-c3ec7a4a33dd None None] MFA Rules not processed for user `57d53facfca6424386b35a42255e8baf`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:01.478729 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-69de7b6f-ee4f-40fd-9df3-c3ec7a4a33dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:01.583921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:01.631516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1186/2358] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:01 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:01.632382 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df96e5e-913a-4769-ad90-8d66b0b8530d service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:01.645594 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df96e5e-913a-4769-ad90-8d66b0b8530d service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:01.653772 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fe89c-8bcb-4d7e-812a-74c08523cd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:01.654625 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fe89c-8bcb-4d7e-812a-74c08523cd8a admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:01.654906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fe89c-8bcb-4d7e-812a-74c08523cd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:01.655278 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8fe89c-8bcb-4d7e-812a-74c08523cd8a admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:01.655861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fe89c-8bcb-4d7e-812a-74c08523cd8a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:01.658096 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8fe89c-8bcb-4d7e-812a-74c08523cd8a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:01.693460 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1173/2359] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:15:01 2022] POST /identity/v3/projects => generated 405 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:15:01.697332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de7b6f-ee4f-40fd-9df3-c3ec7a4a33dd None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:01.708715 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69de7b6f-ee4f-40fd-9df3-c3ec7a4a33dd None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:01.781916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9e78f-a0df-48c6-b8ce-830c69c5f7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:01.782621 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9e78f-a0df-48c6-b8ce-830c69c5f7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:01.782967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9e78f-a0df-48c6-b8ce-830c69c5f7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:01.783302 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a9e78f-a0df-48c6-b8ce-830c69c5f7a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:01.783982 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9e78f-a0df-48c6-b8ce-830c69c5f7a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:01.786083 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a9e78f-a0df-48c6-b8ce-830c69c5f7a4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:01.809239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1187/2360] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:15:01 2022] POST /identity/v3/users => generated 309 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:15:01.813254 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fe89c-8bcb-4d7e-812a-74c08523cd8a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:01.824651 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8fe89c-8bcb-4d7e-812a-74c08523cd8a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:01.829338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3b6a8-3fe0-4fe4-b547-0422af02ec1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:01.829976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3b6a8-3fe0-4fe4-b547-0422af02ec1e admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:01.830253 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3b6a8-3fe0-4fe4-b547-0422af02ec1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:01.830526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3b6a8-3fe0-4fe4-b547-0422af02ec1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:01.831163 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3b6a8-3fe0-4fe4-b547-0422af02ec1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:01.833032 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3b6a8-3fe0-4fe4-b547-0422af02ec1e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:01.844225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1174/2361] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:15:01.848422 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9e78f-a0df-48c6-b8ce-830c69c5f7a4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:01.859239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a9e78f-a0df-48c6-b8ce-830c69c5f7a4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:01.864166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31868ace-29e2-410d-9b41-61611c4d410a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:01.864818 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31868ace-29e2-410d-9b41-61611c4d410a admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:01.865098 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31868ace-29e2-410d-9b41-61611c4d410a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:01.865408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31868ace-29e2-410d-9b41-61611c4d410a admin admin] PATH_INFO: `/v3/projects/12a3e7da835e4efe910164e4d717d416/users/775258db8b074a2aa5611d46fd6dccfe/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:01.880953 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31868ace-29e2-410d-9b41-61611c4d410a admin admin] RBAC: Authorizing `identity:create_grant(project_id=12a3e7da835e4efe910164e4d717d416, user_id=775258db8b074a2aa5611d46fd6dccfe, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:01.883618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31868ace-29e2-410d-9b41-61611c4d410a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:01.897433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1188/2362] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:15:01 2022] PUT /identity/v3/projects/12a3e7da835e4efe910164e4d717d416/users/775258db8b074a2aa5611d46fd6dccfe/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:01.901978 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3b6a8-3fe0-4fe4-b547-0422af02ec1e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:01.912611 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3b6a8-3fe0-4fe4-b547-0422af02ec1e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:01.969646 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882097c7-9295-46ae-b234-a7efff39bc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:01.970257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882097c7-9295-46ae-b234-a7efff39bc22 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:01.970529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882097c7-9295-46ae-b234-a7efff39bc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:01.970793 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882097c7-9295-46ae-b234-a7efff39bc22 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:01.971415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882097c7-9295-46ae-b234-a7efff39bc22 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:01.972982 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882097c7-9295-46ae-b234-a7efff39bc22 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:01.983454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1175/2363] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:15:01.987777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31868ace-29e2-410d-9b41-61611c4d410a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:01.998163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31868ace-29e2-410d-9b41-61611c4d410a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:02.002888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933cbe87-4b81-4456-a11b-b811305b6b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:02.003572 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cbe87-4b81-4456-a11b-b811305b6b09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:02.003843 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cbe87-4b81-4456-a11b-b811305b6b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:02.004106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933cbe87-4b81-4456-a11b-b811305b6b09 admin admin] PATH_INFO: `/v3/projects/12a3e7da835e4efe910164e4d717d416/users/775258db8b074a2aa5611d46fd6dccfe/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:02.005998 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cbe87-4b81-4456-a11b-b811305b6b09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12a3e7da835e4efe910164e4d717d416, user_id=775258db8b074a2aa5611d46fd6dccfe, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:02.008597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933cbe87-4b81-4456-a11b-b811305b6b09 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:02.019296 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1189/2364] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:15:01 2022] PUT /identity/v3/projects/12a3e7da835e4efe910164e4d717d416/users/775258db8b074a2aa5611d46fd6dccfe/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:02.691596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6f87b-2aff-4fe4-b328-800fcd87ff4a None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:02.692182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6f87b-2aff-4fe4-b328-800fcd87ff4a None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:02.692182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe6f87b-2aff-4fe4-b328-800fcd87ff4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:02.723977 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-7fe6f87b-2aff-4fe4-b328-800fcd87ff4a None None] MFA Rules not processed for user `775258db8b074a2aa5611d46fd6dccfe`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:02.725535 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7fe6f87b-2aff-4fe4-b328-800fcd87ff4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:02.850878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1176/2365] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:02 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:04.272790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933cbe87-4b81-4456-a11b-b811305b6b09 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:04.348083 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933cbe87-4b81-4456-a11b-b811305b6b09 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:04.353465 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a471d20-2ae9-4ba0-8f83-2a6568ac701b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:04.354433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a471d20-2ae9-4ba0-8f83-2a6568ac701b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:04.354433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a471d20-2ae9-4ba0-8f83-2a6568ac701b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:04.354721 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a471d20-2ae9-4ba0-8f83-2a6568ac701b admin admin] PATH_INFO: `/v3/users/57d53facfca6424386b35a42255e8baf` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:04.356104 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a471d20-2ae9-4ba0-8f83-2a6568ac701b admin admin] RBAC: Authorizing `identity:delete_user(user_id=57d53facfca6424386b35a42255e8baf)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:04.358364 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a471d20-2ae9-4ba0-8f83-2a6568ac701b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:04.403379 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:04.413092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2a471d20-2ae9-4ba0-8f83-2a6568ac701b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57d53facfca6424386b35a42255e8baf'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:04.421214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2a471d20-2ae9-4ba0-8f83-2a6568ac701b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57d53facfca6424386b35a42255e8baf'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:04.429921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2a471d20-2ae9-4ba0-8f83-2a6568ac701b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57d53facfca6424386b35a42255e8baf'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:04.430996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2a471d20-2ae9-4ba0-8f83-2a6568ac701b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57d53facfca6424386b35a42255e8baf'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:04.443740 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2a471d20-2ae9-4ba0-8f83-2a6568ac701b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57d53facfca6424386b35a42255e8baf'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:04.450379 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1190/2366] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:15:04 2022] DELETE /identity/v3/users/57d53facfca6424386b35a42255e8baf => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:04.626929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6f87b-2aff-4fe4-b328-800fcd87ff4a None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:04.643766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe6f87b-2aff-4fe4-b328-800fcd87ff4a None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:04.685687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:04.715315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374f391-6973-451a-827d-bd83eeaf5220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:04.716224 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374f391-6973-451a-827d-bd83eeaf5220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:04.716577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374f391-6973-451a-827d-bd83eeaf5220 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:04.716925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374f391-6973-451a-827d-bd83eeaf5220 admin admin] PATH_INFO: `/v3/projects/61a4fb2c4c654a9db5c64b9d7071851a` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:04.718151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374f391-6973-451a-827d-bd83eeaf5220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61a4fb2c4c654a9db5c64b9d7071851a)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:04.720054 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374f391-6973-451a-827d-bd83eeaf5220 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:04.772098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0374f391-6973-451a-827d-bd83eeaf5220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a4fb2c4c654a9db5c64b9d7071851a', 'request_id': 'req-0374f391-6973-451a-827d-bd83eeaf5220'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:04.779681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0374f391-6973-451a-827d-bd83eeaf5220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a4fb2c4c654a9db5c64b9d7071851a', 'request_id': 'req-0374f391-6973-451a-827d-bd83eeaf5220'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:04.785405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0374f391-6973-451a-827d-bd83eeaf5220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a4fb2c4c654a9db5c64b9d7071851a', 'request_id': 'req-0374f391-6973-451a-827d-bd83eeaf5220'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:04.796751 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0374f391-6973-451a-827d-bd83eeaf5220 admin admin] The token cache is being invalidate because project 61a4fb2c4c654a9db5c64b9d7071851a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:15:04.797031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0374f391-6973-451a-827d-bd83eeaf5220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:04.797680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0374f391-6973-451a-827d-bd83eeaf5220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:04.799741 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1177/2367] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:15:04 2022] DELETE /identity/v3/projects/61a4fb2c4c654a9db5c64b9d7071851a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:04.804082 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a471d20-2ae9-4ba0-8f83-2a6568ac701b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:04.805486 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2a471d20-2ae9-4ba0-8f83-2a6568ac701b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:04.825067 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a471d20-2ae9-4ba0-8f83-2a6568ac701b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:04.884648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843d8442-37d9-4f89-af76-3d91b3c87fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:04.885360 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d8442-37d9-4f89-af76-3d91b3c87fc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:04.885634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d8442-37d9-4f89-af76-3d91b3c87fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:04.885899 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843d8442-37d9-4f89-af76-3d91b3c87fc9 admin admin] PATH_INFO: `/v3/users/775258db8b074a2aa5611d46fd6dccfe` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:04.887013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d8442-37d9-4f89-af76-3d91b3c87fc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=775258db8b074a2aa5611d46fd6dccfe)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:04.888899 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843d8442-37d9-4f89-af76-3d91b3c87fc9 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:04.936676 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-843d8442-37d9-4f89-af76-3d91b3c87fc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '775258db8b074a2aa5611d46fd6dccfe'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:04.943458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-843d8442-37d9-4f89-af76-3d91b3c87fc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '775258db8b074a2aa5611d46fd6dccfe'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:04.951720 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-843d8442-37d9-4f89-af76-3d91b3c87fc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '775258db8b074a2aa5611d46fd6dccfe'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:04.952329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-843d8442-37d9-4f89-af76-3d91b3c87fc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '775258db8b074a2aa5611d46fd6dccfe'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:04.964777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-843d8442-37d9-4f89-af76-3d91b3c87fc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '775258db8b074a2aa5611d46fd6dccfe'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:04.970761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1191/2368] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:15:04 2022] DELETE /identity/v3/users/775258db8b074a2aa5611d46fd6dccfe => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:05.148605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374f391-6973-451a-827d-bd83eeaf5220 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:05.150431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0374f391-6973-451a-827d-bd83eeaf5220 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:05.168263 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374f391-6973-451a-827d-bd83eeaf5220 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:05.239121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22762879-03e8-4690-bfd9-105ba250dae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:05.239967 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22762879-03e8-4690-bfd9-105ba250dae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:05.240306 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22762879-03e8-4690-bfd9-105ba250dae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:05.241417 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22762879-03e8-4690-bfd9-105ba250dae0 admin admin] PATH_INFO: `/v3/projects/12a3e7da835e4efe910164e4d717d416` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:05.242814 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22762879-03e8-4690-bfd9-105ba250dae0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12a3e7da835e4efe910164e4d717d416)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:05.244860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22762879-03e8-4690-bfd9-105ba250dae0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:05.300732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-22762879-03e8-4690-bfd9-105ba250dae0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12a3e7da835e4efe910164e4d717d416', 'request_id': 'req-22762879-03e8-4690-bfd9-105ba250dae0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:05.310930 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-22762879-03e8-4690-bfd9-105ba250dae0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12a3e7da835e4efe910164e4d717d416', 'request_id': 'req-22762879-03e8-4690-bfd9-105ba250dae0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:05.316812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-22762879-03e8-4690-bfd9-105ba250dae0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12a3e7da835e4efe910164e4d717d416', 'request_id': 'req-22762879-03e8-4690-bfd9-105ba250dae0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:05.318074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:05.330047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-22762879-03e8-4690-bfd9-105ba250dae0 admin admin] The token cache is being invalidate because project 12a3e7da835e4efe910164e4d717d416 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:15:05.330571 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-22762879-03e8-4690-bfd9-105ba250dae0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:05.331262 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-22762879-03e8-4690-bfd9-105ba250dae0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:05.333020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1178/2369] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:15:05 2022] DELETE /identity/v3/projects/12a3e7da835e4efe910164e4d717d416 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:05.338489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c29bc-0046-43e6-851e-b073281de68c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:05.338978 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c29bc-0046-43e6-851e-b073281de68c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:05.339084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c29bc-0046-43e6-851e-b073281de68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:05.361671 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-640c29bc-0046-43e6-851e-b073281de68c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:05.362882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-640c29bc-0046-43e6-851e-b073281de68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:05.393315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:05.433778 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1192/2370] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:05.435404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4768f-d7a1-44d1-bdfc-882fd5f3a56b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:05.435758 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4768f-d7a1-44d1-bdfc-882fd5f3a56b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:05.436098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab4768f-d7a1-44d1-bdfc-882fd5f3a56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:05.456768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9ab4768f-d7a1-44d1-bdfc-882fd5f3a56b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:05.458114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9ab4768f-d7a1-44d1-bdfc-882fd5f3a56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:05.468355 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1179/2371] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:05.472265 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c29bc-0046-43e6-851e-b073281de68c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:05.483187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c29bc-0046-43e6-851e-b073281de68c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:05.493387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9cc0e7-13ac-4ffa-b1c6-358aac7d6e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:05.494017 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cc0e7-13ac-4ffa-b1c6-358aac7d6e29 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:05.494301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cc0e7-13ac-4ffa-b1c6-358aac7d6e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:05.494705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cc0e7-13ac-4ffa-b1c6-358aac7d6e29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:05.495222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9cc0e7-13ac-4ffa-b1c6-358aac7d6e29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:15:05.495528 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9cc0e7-13ac-4ffa-b1c6-358aac7d6e29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:05.497166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9cc0e7-13ac-4ffa-b1c6-358aac7d6e29 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:05.513470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1193/2372] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:15:05.517870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4768f-d7a1-44d1-bdfc-882fd5f3a56b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:05.531091 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab4768f-d7a1-44d1-bdfc-882fd5f3a56b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:05.536134 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51b0f8-caf5-498f-a020-4fa405746406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:05.536740 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51b0f8-caf5-498f-a020-4fa405746406 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:05.537019 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51b0f8-caf5-498f-a020-4fa405746406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:05.537335 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51b0f8-caf5-498f-a020-4fa405746406 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:05.537980 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51b0f8-caf5-498f-a020-4fa405746406 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:05.539877 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db51b0f8-caf5-498f-a020-4fa405746406 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:05.568692 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1180/2373] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:15:05 2022] POST /identity/v3/projects => generated 409 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:15:05.573446 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9cc0e7-13ac-4ffa-b1c6-358aac7d6e29 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:05.583374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9cc0e7-13ac-4ffa-b1c6-358aac7d6e29 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:05.648356 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfffe6-741b-4c62-8f94-d820ab3d0835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:05.648968 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfffe6-741b-4c62-8f94-d820ab3d0835 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:05.649326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfffe6-741b-4c62-8f94-d820ab3d0835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:05.649627 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bfffe6-741b-4c62-8f94-d820ab3d0835 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:05.650802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfffe6-741b-4c62-8f94-d820ab3d0835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:05.652739 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bfffe6-741b-4c62-8f94-d820ab3d0835 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:05.673231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1194/2374] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:15:05 2022] POST /identity/v3/users => generated 305 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:15:05.677300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51b0f8-caf5-498f-a020-4fa405746406 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:05.687519 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51b0f8-caf5-498f-a020-4fa405746406 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:05.692089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fba6ea-1644-4aed-87bd-650bc3c5ff26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:05.692677 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fba6ea-1644-4aed-87bd-650bc3c5ff26 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:05.692953 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fba6ea-1644-4aed-87bd-650bc3c5ff26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:05.693257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fba6ea-1644-4aed-87bd-650bc3c5ff26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:05.693892 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fba6ea-1644-4aed-87bd-650bc3c5ff26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:05.695472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fba6ea-1644-4aed-87bd-650bc3c5ff26 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:05.706598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1181/2375] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:15:05.710636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfffe6-741b-4c62-8f94-d820ab3d0835 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:05.721436 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bfffe6-741b-4c62-8f94-d820ab3d0835 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:05.726019 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad96f6d-5795-4a18-a6b5-57b63d23583b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:05.726607 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad96f6d-5795-4a18-a6b5-57b63d23583b admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:05.726881 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad96f6d-5795-4a18-a6b5-57b63d23583b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:05.727150 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad96f6d-5795-4a18-a6b5-57b63d23583b admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:05.727799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad96f6d-5795-4a18-a6b5-57b63d23583b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:05.729404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad96f6d-5795-4a18-a6b5-57b63d23583b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:05.739915 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1195/2376] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:15:05.744157 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fba6ea-1644-4aed-87bd-650bc3c5ff26 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:05.754083 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fba6ea-1644-4aed-87bd-650bc3c5ff26 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:05.758730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6ceb0-8d5a-4bbb-ae98-25b5cfebe18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:05.759918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6ceb0-8d5a-4bbb-ae98-25b5cfebe18e admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:05.760196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6ceb0-8d5a-4bbb-ae98-25b5cfebe18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:05.760494 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6ceb0-8d5a-4bbb-ae98-25b5cfebe18e admin admin] PATH_INFO: `/v3/projects/c12dcb731c7140b0a0f77f0dbaa96b1b/users/198f2a6cc8b94086ac8744d2a38b318d/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:05.776632 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6ceb0-8d5a-4bbb-ae98-25b5cfebe18e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c12dcb731c7140b0a0f77f0dbaa96b1b, user_id=198f2a6cc8b94086ac8744d2a38b318d, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:05.779315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed6ceb0-8d5a-4bbb-ae98-25b5cfebe18e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:05.791825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1182/2377] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:15:05 2022] PUT /identity/v3/projects/c12dcb731c7140b0a0f77f0dbaa96b1b/users/198f2a6cc8b94086ac8744d2a38b318d/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:05.803437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e182f-2304-4f3d-b82e-82c6a7d6e59c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:05.803727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e182f-2304-4f3d-b82e-82c6a7d6e59c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:05.804003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e182f-2304-4f3d-b82e-82c6a7d6e59c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:05.832032 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8d5e182f-2304-4f3d-b82e-82c6a7d6e59c None None] MFA Rules not processed for user `198f2a6cc8b94086ac8744d2a38b318d`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:05.833400 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8d5e182f-2304-4f3d-b82e-82c6a7d6e59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:05.898320 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1196/2378] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:05 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:05.903472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6ceb0-8d5a-4bbb-ae98-25b5cfebe18e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:05.969300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6ceb0-8d5a-4bbb-ae98-25b5cfebe18e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:05.974128 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce20b0e-29b8-46b4-b533-b521bd38663b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:05.974726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce20b0e-29b8-46b4-b533-b521bd38663b admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:05.975001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce20b0e-29b8-46b4-b533-b521bd38663b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:05.975268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce20b0e-29b8-46b4-b533-b521bd38663b admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:05.975891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce20b0e-29b8-46b4-b533-b521bd38663b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:05.977798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce20b0e-29b8-46b4-b533-b521bd38663b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:06.007349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1183/2379] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:15:05 2022] POST /identity/v3/projects => generated 405 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:15:06.010506 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e182f-2304-4f3d-b82e-82c6a7d6e59c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:06.021386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e182f-2304-4f3d-b82e-82c6a7d6e59c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:06.047559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:06.081078 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a48e04-f6bb-4c25-9a7d-e7c7237a18dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:06.081782 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a48e04-f6bb-4c25-9a7d-e7c7237a18dc admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:06.082068 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a48e04-f6bb-4c25-9a7d-e7c7237a18dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:06.082339 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a48e04-f6bb-4c25-9a7d-e7c7237a18dc admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:06.082956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a48e04-f6bb-4c25-9a7d-e7c7237a18dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:06.084875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a48e04-f6bb-4c25-9a7d-e7c7237a18dc admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:06.106709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1197/2380] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:15:06 2022] POST /identity/v3/users => generated 309 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:15:06.111058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce20b0e-29b8-46b4-b533-b521bd38663b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:06.121873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce20b0e-29b8-46b4-b533-b521bd38663b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:06.126519 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6ba2a-baff-4a09-ae3a-a0082cf0a8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:06.127108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6ba2a-baff-4a09-ae3a-a0082cf0a8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:06.127384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6ba2a-baff-4a09-ae3a-a0082cf0a8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:06.127650 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f6ba2a-baff-4a09-ae3a-a0082cf0a8b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:06.128308 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6ba2a-baff-4a09-ae3a-a0082cf0a8b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:06.129941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f6ba2a-baff-4a09-ae3a-a0082cf0a8b9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:06.141336 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1184/2381] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:15:06.145497 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a48e04-f6bb-4c25-9a7d-e7c7237a18dc admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:06.155416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a48e04-f6bb-4c25-9a7d-e7c7237a18dc admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:06.159982 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62db9f-03af-4c45-8697-12e0e042b7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:06.160626 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62db9f-03af-4c45-8697-12e0e042b7fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:06.160905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62db9f-03af-4c45-8697-12e0e042b7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:06.161179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac62db9f-03af-4c45-8697-12e0e042b7fd admin admin] PATH_INFO: `/v3/projects/7267cfcd7eac40fd80fffac3bf6e1de8/users/dc3162d86f1c49cc9fdfeda2f8e912d5/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:06.175671 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62db9f-03af-4c45-8697-12e0e042b7fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7267cfcd7eac40fd80fffac3bf6e1de8, user_id=dc3162d86f1c49cc9fdfeda2f8e912d5, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:06.178378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac62db9f-03af-4c45-8697-12e0e042b7fd admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:06.193970 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1198/2382] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:15:06 2022] PUT /identity/v3/projects/7267cfcd7eac40fd80fffac3bf6e1de8/users/dc3162d86f1c49cc9fdfeda2f8e912d5/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:06.200220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6ba2a-baff-4a09-ae3a-a0082cf0a8b9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:06.211594 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f6ba2a-baff-4a09-ae3a-a0082cf0a8b9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:06.273129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ea27f-8aea-4250-aa1d-f4a4024c9334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:06.273873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ea27f-8aea-4250-aa1d-f4a4024c9334 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:06.274307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ea27f-8aea-4250-aa1d-f4a4024c9334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:06.275155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ea27f-8aea-4250-aa1d-f4a4024c9334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:06.275906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ea27f-8aea-4250-aa1d-f4a4024c9334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:06.283200 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ea27f-8aea-4250-aa1d-f4a4024c9334 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:06.286939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:06.293474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1185/2383] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:15:06.297821 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62db9f-03af-4c45-8697-12e0e042b7fd admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:06.316277 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac62db9f-03af-4c45-8697-12e0e042b7fd admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:06.322823 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062975c-e2de-4a1b-8996-e2949fffd7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:06.323614 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062975c-e2de-4a1b-8996-e2949fffd7b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:06.323984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062975c-e2de-4a1b-8996-e2949fffd7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:06.324688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062975c-e2de-4a1b-8996-e2949fffd7b3 admin admin] PATH_INFO: `/v3/projects/7267cfcd7eac40fd80fffac3bf6e1de8/users/dc3162d86f1c49cc9fdfeda2f8e912d5/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:06.326588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062975c-e2de-4a1b-8996-e2949fffd7b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7267cfcd7eac40fd80fffac3bf6e1de8, user_id=dc3162d86f1c49cc9fdfeda2f8e912d5, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:06.329839 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062975c-e2de-4a1b-8996-e2949fffd7b3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:06.342835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1199/2384] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:15:06 2022] PUT /identity/v3/projects/7267cfcd7eac40fd80fffac3bf6e1de8/users/dc3162d86f1c49cc9fdfeda2f8e912d5/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:15:06.352630 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecca3c-ca64-4a42-9890-936bc3bd8fa7 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:06.353475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecca3c-ca64-4a42-9890-936bc3bd8fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:06.353753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ecca3c-ca64-4a42-9890-936bc3bd8fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:06.394515 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-e4ecca3c-ca64-4a42-9890-936bc3bd8fa7 None None] MFA Rules not processed for user `dc3162d86f1c49cc9fdfeda2f8e912d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:06.395775 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e4ecca3c-ca64-4a42-9890-936bc3bd8fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:06.470067 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1186/2385] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:06 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:06.481674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062975c-e2de-4a1b-8996-e2949fffd7b3 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:06.483373 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4062975c-e2de-4a1b-8996-e2949fffd7b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:06.545101 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062975c-e2de-4a1b-8996-e2949fffd7b3 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:06.548876 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183a721-6869-49a6-84ca-7d68c21389a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:06.549441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183a721-6869-49a6-84ca-7d68c21389a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:06.549717 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183a721-6869-49a6-84ca-7d68c21389a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:06.549986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b183a721-6869-49a6-84ca-7d68c21389a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:06.560045 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183a721-6869-49a6-84ca-7d68c21389a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:06.561496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b183a721-6869-49a6-84ca-7d68c21389a9 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:06.571563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1200/2386] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:06 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:15:06.792339 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ecca3c-ca64-4a42-9890-936bc3bd8fa7 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:06.802890 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ecca3c-ca64-4a42-9890-936bc3bd8fa7 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:06.807131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb6744-ea0a-4099-97eb-919e4a8a1d61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:06.807756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb6744-ea0a-4099-97eb-919e4a8a1d61 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:06.808034 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb6744-ea0a-4099-97eb-919e4a8a1d61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:06.808306 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb6744-ea0a-4099-97eb-919e4a8a1d61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:06.884463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb6744-ea0a-4099-97eb-919e4a8a1d61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:06.886016 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb6744-ea0a-4099-97eb-919e4a8a1d61 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:06.895369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1187/2387] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:06 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:16.566724 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=57,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:16.891069 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=62 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:27.567361 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183a721-6869-49a6-84ca-7d68c21389a9 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:27.570299 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b183a721-6869-49a6-84ca-7d68c21389a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:27.657648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b183a721-6869-49a6-84ca-7d68c21389a9 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:27.662670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735b2b3-e2f5-4571-86d6-10c5d890c427 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:27.663274 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735b2b3-e2f5-4571-86d6-10c5d890c427 service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:27.663551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735b2b3-e2f5-4571-86d6-10c5d890c427 service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:27.663821 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735b2b3-e2f5-4571-86d6-10c5d890c427 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:27.666173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2735b2b3-e2f5-4571-86d6-10c5d890c427 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:27.679087 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735b2b3-e2f5-4571-86d6-10c5d890c427 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:27.680680 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735b2b3-e2f5-4571-86d6-10c5d890c427 service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:27.692381 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1201/2388] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:28.800506 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb6744-ea0a-4099-97eb-919e4a8a1d61 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:28.863639 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb6744-ea0a-4099-97eb-919e4a8a1d61 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:28.868150 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e034e-1567-4d3b-99a3-c6d4c08d7b6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:28.868767 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e034e-1567-4d3b-99a3-c6d4c08d7b6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:28.869609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e034e-1567-4d3b-99a3-c6d4c08d7b6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:28.869879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e034e-1567-4d3b-99a3-c6d4c08d7b6b admin admin] PATH_INFO: `/v3/users/198f2a6cc8b94086ac8744d2a38b318d` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:28.871013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e034e-1567-4d3b-99a3-c6d4c08d7b6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=198f2a6cc8b94086ac8744d2a38b318d)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:28.872905 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e034e-1567-4d3b-99a3-c6d4c08d7b6b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:28.917604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a85e034e-1567-4d3b-99a3-c6d4c08d7b6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '198f2a6cc8b94086ac8744d2a38b318d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:28.924452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a85e034e-1567-4d3b-99a3-c6d4c08d7b6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '198f2a6cc8b94086ac8744d2a38b318d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:28.937094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a85e034e-1567-4d3b-99a3-c6d4c08d7b6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '198f2a6cc8b94086ac8744d2a38b318d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:28.937809 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a85e034e-1567-4d3b-99a3-c6d4c08d7b6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '198f2a6cc8b94086ac8744d2a38b318d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:28.938395 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a85e034e-1567-4d3b-99a3-c6d4c08d7b6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '198f2a6cc8b94086ac8744d2a38b318d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:28.953004 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1188/2389] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:15:28 2022] DELETE /identity/v3/users/198f2a6cc8b94086ac8744d2a38b318d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:28.957666 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390604e-ceb6-4408-93c6-26fbafaf210a None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:28.958144 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390604e-ceb6-4408-93c6-26fbafaf210a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:28.958239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2390604e-ceb6-4408-93c6-26fbafaf210a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:28.977092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2390604e-ceb6-4408-93c6-26fbafaf210a None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:28.978251 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2390604e-ceb6-4408-93c6-26fbafaf210a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:29.038160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1202/2390] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:15:29.047808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e034e-1567-4d3b-99a3-c6d4c08d7b6b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:29.049557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a85e034e-1567-4d3b-99a3-c6d4c08d7b6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:29.107337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e034e-1567-4d3b-99a3-c6d4c08d7b6b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:29.111007 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6f3ef-0078-4867-a0f8-686fa9484a2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:29.111603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6f3ef-0078-4867-a0f8-686fa9484a2b service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:29.111880 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6f3ef-0078-4867-a0f8-686fa9484a2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:29.112151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae6f3ef-0078-4867-a0f8-686fa9484a2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:29.121571 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6f3ef-0078-4867-a0f8-686fa9484a2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:29.123032 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae6f3ef-0078-4867-a0f8-686fa9484a2b service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:29.132852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1189/2391] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:29.404066 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390604e-ceb6-4408-93c6-26fbafaf210a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:29.405717 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2390604e-ceb6-4408-93c6-26fbafaf210a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:29.419398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2390604e-ceb6-4408-93c6-26fbafaf210a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:29.424262 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ee420-9851-42f5-98fe-3f48f820f590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:29.425025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ee420-9851-42f5-98fe-3f48f820f590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:29.425353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ee420-9851-42f5-98fe-3f48f820f590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:29.425653 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657ee420-9851-42f5-98fe-3f48f820f590 admin admin] PATH_INFO: `/v3/projects/c12dcb731c7140b0a0f77f0dbaa96b1b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:29.426750 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ee420-9851-42f5-98fe-3f48f820f590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c12dcb731c7140b0a0f77f0dbaa96b1b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:29.428663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657ee420-9851-42f5-98fe-3f48f820f590 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:29.485987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-657ee420-9851-42f5-98fe-3f48f820f590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c12dcb731c7140b0a0f77f0dbaa96b1b', 'request_id': 'req-657ee420-9851-42f5-98fe-3f48f820f590'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:29.492677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-657ee420-9851-42f5-98fe-3f48f820f590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c12dcb731c7140b0a0f77f0dbaa96b1b', 'request_id': 'req-657ee420-9851-42f5-98fe-3f48f820f590'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:29.499056 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-657ee420-9851-42f5-98fe-3f48f820f590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c12dcb731c7140b0a0f77f0dbaa96b1b', 'request_id': 'req-657ee420-9851-42f5-98fe-3f48f820f590'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:29.512008 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-657ee420-9851-42f5-98fe-3f48f820f590 admin admin] The token cache is being invalidate because project c12dcb731c7140b0a0f77f0dbaa96b1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:15:29.512299 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-657ee420-9851-42f5-98fe-3f48f820f590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:29.512912 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-657ee420-9851-42f5-98fe-3f48f820f590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:29.514448 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1203/2392] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:15:29 2022] DELETE /identity/v3/projects/c12dcb731c7140b0a0f77f0dbaa96b1b => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:29.518831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6f3ef-0078-4867-a0f8-686fa9484a2b service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:29.528621 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae6f3ef-0078-4867-a0f8-686fa9484a2b service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:29.546836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:29.586017 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184fa69a-a8c1-4395-bc82-2caa36075ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:29.586645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fa69a-a8c1-4395-bc82-2caa36075ff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:29.586922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fa69a-a8c1-4395-bc82-2caa36075ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:29.587197 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fa69a-a8c1-4395-bc82-2caa36075ff3 admin admin] PATH_INFO: `/v3/users/dc3162d86f1c49cc9fdfeda2f8e912d5` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:29.588826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fa69a-a8c1-4395-bc82-2caa36075ff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc3162d86f1c49cc9fdfeda2f8e912d5)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:29.590764 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184fa69a-a8c1-4395-bc82-2caa36075ff3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:29.651838 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-184fa69a-a8c1-4395-bc82-2caa36075ff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc3162d86f1c49cc9fdfeda2f8e912d5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:29.658969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-184fa69a-a8c1-4395-bc82-2caa36075ff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc3162d86f1c49cc9fdfeda2f8e912d5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:29.672031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-184fa69a-a8c1-4395-bc82-2caa36075ff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc3162d86f1c49cc9fdfeda2f8e912d5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:29.672866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-184fa69a-a8c1-4395-bc82-2caa36075ff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc3162d86f1c49cc9fdfeda2f8e912d5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:29.673582 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-184fa69a-a8c1-4395-bc82-2caa36075ff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc3162d86f1c49cc9fdfeda2f8e912d5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:29.687053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1190/2393] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:15:29 2022] DELETE /identity/v3/users/dc3162d86f1c49cc9fdfeda2f8e912d5 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:15:30.024509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ee420-9851-42f5-98fe-3f48f820f590 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:30.026624 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-657ee420-9851-42f5-98fe-3f48f820f590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:30.041506 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657ee420-9851-42f5-98fe-3f48f820f590 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:30.053222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:30.104277 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf13ba8-44c8-4f56-a09c-292f44559453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:30.104935 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf13ba8-44c8-4f56-a09c-292f44559453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:30.105603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf13ba8-44c8-4f56-a09c-292f44559453 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:30.105763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf13ba8-44c8-4f56-a09c-292f44559453 admin admin] PATH_INFO: `/v3/projects/7267cfcd7eac40fd80fffac3bf6e1de8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:30.107072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf13ba8-44c8-4f56-a09c-292f44559453 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7267cfcd7eac40fd80fffac3bf6e1de8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:30.108952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf13ba8-44c8-4f56-a09c-292f44559453 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:30.154538 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ddf13ba8-44c8-4f56-a09c-292f44559453 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7267cfcd7eac40fd80fffac3bf6e1de8', 'request_id': 'req-ddf13ba8-44c8-4f56-a09c-292f44559453'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:30.161231 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ddf13ba8-44c8-4f56-a09c-292f44559453 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7267cfcd7eac40fd80fffac3bf6e1de8', 'request_id': 'req-ddf13ba8-44c8-4f56-a09c-292f44559453'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:30.166748 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ddf13ba8-44c8-4f56-a09c-292f44559453 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7267cfcd7eac40fd80fffac3bf6e1de8', 'request_id': 'req-ddf13ba8-44c8-4f56-a09c-292f44559453'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:30.179816 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ddf13ba8-44c8-4f56-a09c-292f44559453 admin admin] The token cache is being invalidate because project 7267cfcd7eac40fd80fffac3bf6e1de8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:15:30.180106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ddf13ba8-44c8-4f56-a09c-292f44559453 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:30.180745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ddf13ba8-44c8-4f56-a09c-292f44559453 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:30.182370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1204/2394] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:15:30 2022] DELETE /identity/v3/projects/7267cfcd7eac40fd80fffac3bf6e1de8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:30.187559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f8926-b3dd-4f97-8652-bb3473eeb8a4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:30.188024 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f8926-b3dd-4f97-8652-bb3473eeb8a4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:30.188251 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f8926-b3dd-4f97-8652-bb3473eeb8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:30.208638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-1c2f8926-b3dd-4f97-8652-bb3473eeb8a4 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:30.209871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1c2f8926-b3dd-4f97-8652-bb3473eeb8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:30.281257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1191/2395] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:30.288843 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59258d-9398-49d5-868e-185484104bf9 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:30.289133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59258d-9398-49d5-868e-185484104bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:30.289458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d59258d-9398-49d5-868e-185484104bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:30.310292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2d59258d-9398-49d5-868e-185484104bf9 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:30.311500 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2d59258d-9398-49d5-868e-185484104bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:30.319361 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1205/2396] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:30.324164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f8926-b3dd-4f97-8652-bb3473eeb8a4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:30.333734 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f8926-b3dd-4f97-8652-bb3473eeb8a4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:30.338671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fb195-a26b-4865-b382-6a940e435fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:30.339257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fb195-a26b-4865-b382-6a940e435fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:30.339531 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fb195-a26b-4865-b382-6a940e435fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:30.339796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1fb195-a26b-4865-b382-6a940e435fb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:30.340434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fb195-a26b-4865-b382-6a940e435fb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:15:30.340727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fb195-a26b-4865-b382-6a940e435fb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:30.342392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1fb195-a26b-4865-b382-6a940e435fb3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:30.354392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1192/2397] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:15:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:15:30.359394 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59258d-9398-49d5-868e-185484104bf9 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:30.368654 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d59258d-9398-49d5-868e-185484104bf9 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:30.374292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbce52-a748-4237-9eae-efa3d994a292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:30.374885 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbce52-a748-4237-9eae-efa3d994a292 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:30.375163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbce52-a748-4237-9eae-efa3d994a292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:30.375431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dbce52-a748-4237-9eae-efa3d994a292 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:30.376059 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbce52-a748-4237-9eae-efa3d994a292 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:30.377980 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dbce52-a748-4237-9eae-efa3d994a292 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:30.404952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1206/2398] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:15:30 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:15:30.409156 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fb195-a26b-4865-b382-6a940e435fb3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:30.419014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1fb195-a26b-4865-b382-6a940e435fb3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:30.443922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:30.475695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e10a5-c899-410f-a9fa-1ca89e610b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:30.476283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e10a5-c899-410f-a9fa-1ca89e610b32 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:30.476553 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e10a5-c899-410f-a9fa-1ca89e610b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:30.476820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884e10a5-c899-410f-a9fa-1ca89e610b32 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:30.477485 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e10a5-c899-410f-a9fa-1ca89e610b32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:30.479358 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884e10a5-c899-410f-a9fa-1ca89e610b32 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:30.501565 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1193/2399] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:15:30 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:15:30.505605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbce52-a748-4237-9eae-efa3d994a292 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:30.516786 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dbce52-a748-4237-9eae-efa3d994a292 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:30.521493 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14391e9-37fa-4d3a-a8eb-69c91e309770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:30.522087 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14391e9-37fa-4d3a-a8eb-69c91e309770 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:30.522385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14391e9-37fa-4d3a-a8eb-69c91e309770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:30.522653 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14391e9-37fa-4d3a-a8eb-69c91e309770 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:30.523278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14391e9-37fa-4d3a-a8eb-69c91e309770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:30.524913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14391e9-37fa-4d3a-a8eb-69c91e309770 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:30.534855 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1207/2400] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:15:30.538615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e10a5-c899-410f-a9fa-1ca89e610b32 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:30.549485 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884e10a5-c899-410f-a9fa-1ca89e610b32 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:30.554225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef10f2ee-4eea-46be-9ec2-f1c01347d139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:30.554821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10f2ee-4eea-46be-9ec2-f1c01347d139 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:30.555104 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10f2ee-4eea-46be-9ec2-f1c01347d139 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:30.555375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef10f2ee-4eea-46be-9ec2-f1c01347d139 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:30.556010 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10f2ee-4eea-46be-9ec2-f1c01347d139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:30.558242 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef10f2ee-4eea-46be-9ec2-f1c01347d139 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:30.568962 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1194/2401] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:15:30.573884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14391e9-37fa-4d3a-a8eb-69c91e309770 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:30.583988 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14391e9-37fa-4d3a-a8eb-69c91e309770 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:30.590384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c895a8-c693-445c-bd47-901a059a9303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:30.590384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c895a8-c693-445c-bd47-901a059a9303 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:30.590384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c895a8-c693-445c-bd47-901a059a9303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:30.590384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c895a8-c693-445c-bd47-901a059a9303 admin admin] PATH_INFO: `/v3/projects/f778230a9266406ea97e64c85a45d832/users/58ca88f0707a46b191aa8def5550f79e/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:30.605514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c895a8-c693-445c-bd47-901a059a9303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f778230a9266406ea97e64c85a45d832, user_id=58ca88f0707a46b191aa8def5550f79e, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:30.608124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c895a8-c693-445c-bd47-901a059a9303 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:30.622776 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1208/2402] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:15:30 2022] PUT /identity/v3/projects/f778230a9266406ea97e64c85a45d832/users/58ca88f0707a46b191aa8def5550f79e/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:30.635354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8849f2-e9ec-49e5-afa3-985b6d643d69 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:30.635637 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8849f2-e9ec-49e5-afa3-985b6d643d69 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:30.635910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8849f2-e9ec-49e5-afa3-985b6d643d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:30.667829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3e8849f2-e9ec-49e5-afa3-985b6d643d69 None None] MFA Rules not processed for user `58ca88f0707a46b191aa8def5550f79e`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:30.669061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3e8849f2-e9ec-49e5-afa3-985b6d643d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:30.747608 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1195/2403] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:30.758533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c895a8-c693-445c-bd47-901a059a9303 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:30.760757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-51c895a8-c693-445c-bd47-901a059a9303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:30.809260 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:30.825222 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c895a8-c693-445c-bd47-901a059a9303 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:30.829098 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1112a4-be6d-424f-9361-f96d65b1e15c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:30.829939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1112a4-be6d-424f-9361-f96d65b1e15c service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:30.830216 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1112a4-be6d-424f-9361-f96d65b1e15c service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:30.830483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1112a4-be6d-424f-9361-f96d65b1e15c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:30.839526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1112a4-be6d-424f-9361-f96d65b1e15c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:30.840943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1112a4-be6d-424f-9361-f96d65b1e15c service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:30.850224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1209/2404] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:30.877782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8849f2-e9ec-49e5-afa3-985b6d643d69 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:30.987031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8849f2-e9ec-49e5-afa3-985b6d643d69 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:30.991753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2c6da-1ca0-4003-bc1d-50b3ad875ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:30.992395 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2c6da-1ca0-4003-bc1d-50b3ad875ce6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:30.992672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2c6da-1ca0-4003-bc1d-50b3ad875ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:30.992946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce2c6da-1ca0-4003-bc1d-50b3ad875ce6 admin admin] PATH_INFO: `/v3/users/58ca88f0707a46b191aa8def5550f79e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:30.994721 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2c6da-1ca0-4003-bc1d-50b3ad875ce6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58ca88f0707a46b191aa8def5550f79e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:30.996621 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce2c6da-1ca0-4003-bc1d-50b3ad875ce6 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:31.045329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:31.055443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7ce2c6da-1ca0-4003-bc1d-50b3ad875ce6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58ca88f0707a46b191aa8def5550f79e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:31.065989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7ce2c6da-1ca0-4003-bc1d-50b3ad875ce6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58ca88f0707a46b191aa8def5550f79e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:31.080940 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7ce2c6da-1ca0-4003-bc1d-50b3ad875ce6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58ca88f0707a46b191aa8def5550f79e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:31.081706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7ce2c6da-1ca0-4003-bc1d-50b3ad875ce6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58ca88f0707a46b191aa8def5550f79e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:31.082316 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7ce2c6da-1ca0-4003-bc1d-50b3ad875ce6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58ca88f0707a46b191aa8def5550f79e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:31.109048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1196/2405] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:15:30 2022] DELETE /identity/v3/users/58ca88f0707a46b191aa8def5550f79e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:31.113534 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3fa45-3da4-470f-86dc-4ffd671e5ea8 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:31.114035 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3fa45-3da4-470f-86dc-4ffd671e5ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:31.114137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3fa45-3da4-470f-86dc-4ffd671e5ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:31.133864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-02c3fa45-3da4-470f-86dc-4ffd671e5ea8 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:31.135161 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-02c3fa45-3da4-470f-86dc-4ffd671e5ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:31.221249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1210/2406] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:15:31.233310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2c6da-1ca0-4003-bc1d-50b3ad875ce6 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:31.234980 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7ce2c6da-1ca0-4003-bc1d-50b3ad875ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:31.302126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce2c6da-1ca0-4003-bc1d-50b3ad875ce6 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:31.305845 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d142ff0b-323c-476f-a943-be87e24dc2ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:31.306418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142ff0b-323c-476f-a943-be87e24dc2ba service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:31.306691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142ff0b-323c-476f-a943-be87e24dc2ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:31.306961 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d142ff0b-323c-476f-a943-be87e24dc2ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:31.316412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142ff0b-323c-476f-a943-be87e24dc2ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:31.317867 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d142ff0b-323c-476f-a943-be87e24dc2ba service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:31.327406 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1197/2407] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:31.655392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3fa45-3da4-470f-86dc-4ffd671e5ea8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:31.656797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-02c3fa45-3da4-470f-86dc-4ffd671e5ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:31.669555 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3fa45-3da4-470f-86dc-4ffd671e5ea8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:31.674644 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9fcf37-2b97-45b9-9719-004cea9794ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:31.675311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fcf37-2b97-45b9-9719-004cea9794ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:31.675606 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fcf37-2b97-45b9-9719-004cea9794ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:31.675865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9fcf37-2b97-45b9-9719-004cea9794ab admin admin] PATH_INFO: `/v3/projects/f778230a9266406ea97e64c85a45d832` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:31.677124 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9fcf37-2b97-45b9-9719-004cea9794ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=f778230a9266406ea97e64c85a45d832)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:31.679055 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9fcf37-2b97-45b9-9719-004cea9794ab admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:31.739960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9f9fcf37-2b97-45b9-9719-004cea9794ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f778230a9266406ea97e64c85a45d832', 'request_id': 'req-9f9fcf37-2b97-45b9-9719-004cea9794ab'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:31.746192 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9f9fcf37-2b97-45b9-9719-004cea9794ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f778230a9266406ea97e64c85a45d832', 'request_id': 'req-9f9fcf37-2b97-45b9-9719-004cea9794ab'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:31.753702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9f9fcf37-2b97-45b9-9719-004cea9794ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f778230a9266406ea97e64c85a45d832', 'request_id': 'req-9f9fcf37-2b97-45b9-9719-004cea9794ab'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:31.773944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9f9fcf37-2b97-45b9-9719-004cea9794ab admin admin] The token cache is being invalidate because project f778230a9266406ea97e64c85a45d832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:15:31.774310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9f9fcf37-2b97-45b9-9719-004cea9794ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:31.775859 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9f9fcf37-2b97-45b9-9719-004cea9794ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:31.777641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1211/2408] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:15:31 2022] DELETE /identity/v3/projects/f778230a9266406ea97e64c85a45d832 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:31.782346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cb5b3-3a2b-4fec-baca-a09aabcacf0e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:31.782545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cb5b3-3a2b-4fec-baca-a09aabcacf0e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:31.782858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18cb5b3-3a2b-4fec-baca-a09aabcacf0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:31.807546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-f18cb5b3-3a2b-4fec-baca-a09aabcacf0e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:31.808678 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f18cb5b3-3a2b-4fec-baca-a09aabcacf0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:31.879296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1198/2409] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:31.894740 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c499e4-6097-4527-af34-27cf91b0e80f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:31.895191 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c499e4-6097-4527-af34-27cf91b0e80f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:31.895817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c499e4-6097-4527-af34-27cf91b0e80f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:31.918299 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f5c499e4-6097-4527-af34-27cf91b0e80f None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:31.919549 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f5c499e4-6097-4527-af34-27cf91b0e80f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:31.927369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1212/2410] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:31.932333 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cb5b3-3a2b-4fec-baca-a09aabcacf0e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:31.942121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18cb5b3-3a2b-4fec-baca-a09aabcacf0e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:31.946809 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec5a42-28db-4543-9709-b72b82380018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:31.948067 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec5a42-28db-4543-9709-b72b82380018 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:31.948315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec5a42-28db-4543-9709-b72b82380018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:31.948696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec5a42-28db-4543-9709-b72b82380018 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:31.949374 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec5a42-28db-4543-9709-b72b82380018 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:15:31.949721 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec5a42-28db-4543-9709-b72b82380018 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:31.951307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec5a42-28db-4543-9709-b72b82380018 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:31.963139 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1199/2411] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:15:31.967479 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c499e4-6097-4527-af34-27cf91b0e80f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:31.976687 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c499e4-6097-4527-af34-27cf91b0e80f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:31.981529 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f16aa-a1f7-403f-b273-57247f94b2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:31.982109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f16aa-a1f7-403f-b273-57247f94b2b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:31.982385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f16aa-a1f7-403f-b273-57247f94b2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:31.982800 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f16aa-a1f7-403f-b273-57247f94b2b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:31.983453 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f16aa-a1f7-403f-b273-57247f94b2b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:31.985355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f16aa-a1f7-403f-b273-57247f94b2b1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:32.021176 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1213/2412] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:15:31 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:15:32.025630 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec5a42-28db-4543-9709-b72b82380018 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:32.038726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec5a42-28db-4543-9709-b72b82380018 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:32.085276 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:32.099153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56a986-cd41-4a34-8ba3-5f382555293d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:32.099835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56a986-cd41-4a34-8ba3-5f382555293d admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:32.100187 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56a986-cd41-4a34-8ba3-5f382555293d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:32.100529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e56a986-cd41-4a34-8ba3-5f382555293d admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:32.101255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56a986-cd41-4a34-8ba3-5f382555293d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:32.103232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e56a986-cd41-4a34-8ba3-5f382555293d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:32.124780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1200/2413] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:15:32 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:15:32.128804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f16aa-a1f7-403f-b273-57247f94b2b1 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:32.139732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f16aa-a1f7-403f-b273-57247f94b2b1 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:32.144703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85499b-d015-476c-abb9-f2acc9fa3c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:32.145328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85499b-d015-476c-abb9-f2acc9fa3c03 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:32.145605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85499b-d015-476c-abb9-f2acc9fa3c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:32.145871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85499b-d015-476c-abb9-f2acc9fa3c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:32.146506 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85499b-d015-476c-abb9-f2acc9fa3c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:32.148094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c85499b-d015-476c-abb9-f2acc9fa3c03 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:32.159154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1214/2414] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:15:32.163233 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56a986-cd41-4a34-8ba3-5f382555293d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:32.173774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e56a986-cd41-4a34-8ba3-5f382555293d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:32.178436 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152a93d-ef36-424b-884c-86241fd7b313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:32.179020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152a93d-ef36-424b-884c-86241fd7b313 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:32.179295 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152a93d-ef36-424b-884c-86241fd7b313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:32.179557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152a93d-ef36-424b-884c-86241fd7b313 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:32.180181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152a93d-ef36-424b-884c-86241fd7b313 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:32.181881 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152a93d-ef36-424b-884c-86241fd7b313 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:32.193068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1201/2415] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:15:32.197370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85499b-d015-476c-abb9-f2acc9fa3c03 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:32.207183 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85499b-d015-476c-abb9-f2acc9fa3c03 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:32.211766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf637b9-7bbe-4bdb-9cb8-ff32af2cb447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:32.212415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf637b9-7bbe-4bdb-9cb8-ff32af2cb447 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:32.212688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf637b9-7bbe-4bdb-9cb8-ff32af2cb447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:32.212954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf637b9-7bbe-4bdb-9cb8-ff32af2cb447 admin admin] PATH_INFO: `/v3/projects/76c798b9277442f99f6971991360a425/users/38c8dfeeba794ef2b4031e03327e31ab/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:32.231065 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf637b9-7bbe-4bdb-9cb8-ff32af2cb447 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c798b9277442f99f6971991360a425, user_id=38c8dfeeba794ef2b4031e03327e31ab, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:32.233809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf637b9-7bbe-4bdb-9cb8-ff32af2cb447 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:32.248487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1215/2416] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:15:32 2022] PUT /identity/v3/projects/76c798b9277442f99f6971991360a425/users/38c8dfeeba794ef2b4031e03327e31ab/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:32.259312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64228990-6b39-4e2b-866f-1d518141c84d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:32.259819 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64228990-6b39-4e2b-866f-1d518141c84d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:32.259939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64228990-6b39-4e2b-866f-1d518141c84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:32.289495 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-64228990-6b39-4e2b-866f-1d518141c84d None None] MFA Rules not processed for user `38c8dfeeba794ef2b4031e03327e31ab`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:32.293279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-64228990-6b39-4e2b-866f-1d518141c84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:32.367462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1202/2417] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:32.379126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf637b9-7bbe-4bdb-9cb8-ff32af2cb447 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:32.380821 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1bf637b9-7bbe-4bdb-9cb8-ff32af2cb447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:32.386209 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:32.439812 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf637b9-7bbe-4bdb-9cb8-ff32af2cb447 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:32.443466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c2992-fa7f-425a-80e5-aed731be519c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:32.444062 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c2992-fa7f-425a-80e5-aed731be519c service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:32.444343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c2992-fa7f-425a-80e5-aed731be519c service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:32.444615 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c2992-fa7f-425a-80e5-aed731be519c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:32.452986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c2992-fa7f-425a-80e5-aed731be519c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:32.454437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c2992-fa7f-425a-80e5-aed731be519c service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:32.463134 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1216/2418] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:32.484645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64228990-6b39-4e2b-866f-1d518141c84d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:32.546337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64228990-6b39-4e2b-866f-1d518141c84d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:32.550900 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70353d86-b46e-4642-bcc4-cbfc2d68122a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:32.551515 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70353d86-b46e-4642-bcc4-cbfc2d68122a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:32.551785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70353d86-b46e-4642-bcc4-cbfc2d68122a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:32.552052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70353d86-b46e-4642-bcc4-cbfc2d68122a admin admin] PATH_INFO: `/v3/users/38c8dfeeba794ef2b4031e03327e31ab` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:32.553157 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70353d86-b46e-4642-bcc4-cbfc2d68122a admin admin] RBAC: Authorizing `identity:delete_user(user_id=38c8dfeeba794ef2b4031e03327e31ab)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:32.555170 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70353d86-b46e-4642-bcc4-cbfc2d68122a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:32.601039 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-70353d86-b46e-4642-bcc4-cbfc2d68122a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38c8dfeeba794ef2b4031e03327e31ab'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:32.608886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-70353d86-b46e-4642-bcc4-cbfc2d68122a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38c8dfeeba794ef2b4031e03327e31ab'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:32.610140 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:32.623814 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-70353d86-b46e-4642-bcc4-cbfc2d68122a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38c8dfeeba794ef2b4031e03327e31ab'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:32.624467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-70353d86-b46e-4642-bcc4-cbfc2d68122a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38c8dfeeba794ef2b4031e03327e31ab'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:32.625090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-70353d86-b46e-4642-bcc4-cbfc2d68122a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38c8dfeeba794ef2b4031e03327e31ab'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:32.640721 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1203/2419] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:15:32 2022] DELETE /identity/v3/users/38c8dfeeba794ef2b4031e03327e31ab => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:32.645121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692e496-f95a-4d54-9404-cd512aaf8a02 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:32.645438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692e496-f95a-4d54-9404-cd512aaf8a02 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:32.645720 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4692e496-f95a-4d54-9404-cd512aaf8a02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:32.665513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4692e496-f95a-4d54-9404-cd512aaf8a02 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:32.667205 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4692e496-f95a-4d54-9404-cd512aaf8a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:32.725610 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1217/2420] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:32.735943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70353d86-b46e-4642-bcc4-cbfc2d68122a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:32.737361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-70353d86-b46e-4642-bcc4-cbfc2d68122a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:32.801133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70353d86-b46e-4642-bcc4-cbfc2d68122a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:32.804796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfec295-ad50-4690-af2c-69763c118fae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:32.805412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfec295-ad50-4690-af2c-69763c118fae service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:32.805688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfec295-ad50-4690-af2c-69763c118fae service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:32.805952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfec295-ad50-4690-af2c-69763c118fae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:32.815273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfec295-ad50-4690-af2c-69763c118fae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:32.816684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfec295-ad50-4690-af2c-69763c118fae service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:32.827350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1204/2421] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:33.143480 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692e496-f95a-4d54-9404-cd512aaf8a02 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:33.144977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4692e496-f95a-4d54-9404-cd512aaf8a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:33.161401 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4692e496-f95a-4d54-9404-cd512aaf8a02 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:33.166037 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3792eed1-b252-4df3-8e0c-b9eb4c74f940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:33.166776 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792eed1-b252-4df3-8e0c-b9eb4c74f940 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:33.167056 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792eed1-b252-4df3-8e0c-b9eb4c74f940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:33.167336 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3792eed1-b252-4df3-8e0c-b9eb4c74f940 admin admin] PATH_INFO: `/v3/projects/76c798b9277442f99f6971991360a425` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:33.168468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792eed1-b252-4df3-8e0c-b9eb4c74f940 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76c798b9277442f99f6971991360a425)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:33.170469 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3792eed1-b252-4df3-8e0c-b9eb4c74f940 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:33.236542 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3792eed1-b252-4df3-8e0c-b9eb4c74f940 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76c798b9277442f99f6971991360a425', 'request_id': 'req-3792eed1-b252-4df3-8e0c-b9eb4c74f940'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:33.244451 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3792eed1-b252-4df3-8e0c-b9eb4c74f940 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76c798b9277442f99f6971991360a425', 'request_id': 'req-3792eed1-b252-4df3-8e0c-b9eb4c74f940'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:33.251736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3792eed1-b252-4df3-8e0c-b9eb4c74f940 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76c798b9277442f99f6971991360a425', 'request_id': 'req-3792eed1-b252-4df3-8e0c-b9eb4c74f940'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:33.266705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3792eed1-b252-4df3-8e0c-b9eb4c74f940 admin admin] The token cache is being invalidate because project 76c798b9277442f99f6971991360a425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:15:33.266998 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3792eed1-b252-4df3-8e0c-b9eb4c74f940 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:33.267630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3792eed1-b252-4df3-8e0c-b9eb4c74f940 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:33.269414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1218/2422] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:15:33 2022] DELETE /identity/v3/projects/76c798b9277442f99f6971991360a425 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:15:33.274144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec10eda-9f35-45e8-95ca-5deab621c661 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:33.274634 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec10eda-9f35-45e8-95ca-5deab621c661 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:33.274721 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec10eda-9f35-45e8-95ca-5deab621c661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:33.296198 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-fec10eda-9f35-45e8-95ca-5deab621c661 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:33.297434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-fec10eda-9f35-45e8-95ca-5deab621c661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:33.413220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1205/2423] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:33.422690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee4701-d97c-4279-808b-ee1d4b22bbac None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:33.422981 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee4701-d97c-4279-808b-ee1d4b22bbac None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:33.423261 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee4701-d97c-4279-808b-ee1d4b22bbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:33.445676 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b1ee4701-d97c-4279-808b-ee1d4b22bbac None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:33.446902 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b1ee4701-d97c-4279-808b-ee1d4b22bbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:33.457734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1219/2424] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:33.471289 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec10eda-9f35-45e8-95ca-5deab621c661 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:33.482867 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec10eda-9f35-45e8-95ca-5deab621c661 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:33.487708 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185644d-2c6f-4792-b33c-e0a5f47091be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:33.488311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185644d-2c6f-4792-b33c-e0a5f47091be admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:33.488593 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185644d-2c6f-4792-b33c-e0a5f47091be admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:33.488866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c185644d-2c6f-4792-b33c-e0a5f47091be admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:33.489547 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185644d-2c6f-4792-b33c-e0a5f47091be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:15:33.489851 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185644d-2c6f-4792-b33c-e0a5f47091be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:33.491603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c185644d-2c6f-4792-b33c-e0a5f47091be admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:33.504965 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1206/2425] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:15:33.509558 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee4701-d97c-4279-808b-ee1d4b22bbac None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:33.520126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee4701-d97c-4279-808b-ee1d4b22bbac None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:33.525847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690973dc-6d8c-46f0-9848-47aa25862cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:33.526576 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690973dc-6d8c-46f0-9848-47aa25862cc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:33.526972 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690973dc-6d8c-46f0-9848-47aa25862cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:33.527281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690973dc-6d8c-46f0-9848-47aa25862cc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:33.527950 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690973dc-6d8c-46f0-9848-47aa25862cc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:33.530711 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690973dc-6d8c-46f0-9848-47aa25862cc0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:33.557961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:33.565206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1220/2426] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:15:33 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:15:33.569559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185644d-2c6f-4792-b33c-e0a5f47091be admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:33.580319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c185644d-2c6f-4792-b33c-e0a5f47091be admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:33.650437 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c00ce1-59ba-4b83-a337-cf7eec80b6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:33.651070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c00ce1-59ba-4b83-a337-cf7eec80b6d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:33.651350 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c00ce1-59ba-4b83-a337-cf7eec80b6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:33.651620 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c00ce1-59ba-4b83-a337-cf7eec80b6d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:33.652255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c00ce1-59ba-4b83-a337-cf7eec80b6d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:33.654525 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c00ce1-59ba-4b83-a337-cf7eec80b6d5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:33.665488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:33.688249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690973dc-6d8c-46f0-9848-47aa25862cc0 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:33.688361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1207/2427] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:15:33 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:15:33.700094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690973dc-6d8c-46f0-9848-47aa25862cc0 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:33.705939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca6210-cf52-46f0-a894-18838b3cab25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:33.706724 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca6210-cf52-46f0-a894-18838b3cab25 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:33.707072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca6210-cf52-46f0-a894-18838b3cab25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:33.707412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca6210-cf52-46f0-a894-18838b3cab25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:33.708157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca6210-cf52-46f0-a894-18838b3cab25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:33.710916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca6210-cf52-46f0-a894-18838b3cab25 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:33.722087 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1221/2428] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:15:33.726636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c00ce1-59ba-4b83-a337-cf7eec80b6d5 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:33.738895 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c00ce1-59ba-4b83-a337-cf7eec80b6d5 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:33.744081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958984e5-6be2-462b-a097-105a3c1d6821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:33.745331 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958984e5-6be2-462b-a097-105a3c1d6821 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:33.745701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958984e5-6be2-462b-a097-105a3c1d6821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:33.746044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958984e5-6be2-462b-a097-105a3c1d6821 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:33.746753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958984e5-6be2-462b-a097-105a3c1d6821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:33.748445 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958984e5-6be2-462b-a097-105a3c1d6821 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:33.761227 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1208/2429] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:15:33.765463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca6210-cf52-46f0-a894-18838b3cab25 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:33.775768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca6210-cf52-46f0-a894-18838b3cab25 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:33.780386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4035b2-8a7a-44eb-ada1-6ec75514973a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:33.781035 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4035b2-8a7a-44eb-ada1-6ec75514973a admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:33.781350 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4035b2-8a7a-44eb-ada1-6ec75514973a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:33.781622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4035b2-8a7a-44eb-ada1-6ec75514973a admin admin] PATH_INFO: `/v3/projects/2713c78885694217b7869d91f83f0e7c/users/60201ffe2dcf460d8d643e8e0b749c03/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:33.796817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4035b2-8a7a-44eb-ada1-6ec75514973a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2713c78885694217b7869d91f83f0e7c, user_id=60201ffe2dcf460d8d643e8e0b749c03, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:33.799473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4035b2-8a7a-44eb-ada1-6ec75514973a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:33.813179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1222/2430] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:15:33 2022] PUT /identity/v3/projects/2713c78885694217b7869d91f83f0e7c/users/60201ffe2dcf460d8d643e8e0b749c03/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:33.823414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdaeb21-2420-46a9-aa0c-77058e61d137 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:33.823695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdaeb21-2420-46a9-aa0c-77058e61d137 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:33.823966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdaeb21-2420-46a9-aa0c-77058e61d137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:33.851372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-abdaeb21-2420-46a9-aa0c-77058e61d137 None None] MFA Rules not processed for user `60201ffe2dcf460d8d643e8e0b749c03`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:33.852935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-abdaeb21-2420-46a9-aa0c-77058e61d137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:33.915410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1209/2431] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:33.927073 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4035b2-8a7a-44eb-ada1-6ec75514973a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:33.928652 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5e4035b2-8a7a-44eb-ada1-6ec75514973a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:33.984211 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4035b2-8a7a-44eb-ada1-6ec75514973a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:33.987814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbf25f-72c9-42fd-b549-850ed57163d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:33.988408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbf25f-72c9-42fd-b549-850ed57163d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:33.988680 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbf25f-72c9-42fd-b549-850ed57163d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:33.988944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbf25f-72c9-42fd-b549-850ed57163d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:33.997834 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbf25f-72c9-42fd-b549-850ed57163d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:33.999303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbf25f-72c9-42fd-b549-850ed57163d8 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:34.009134 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1223/2432] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:38.141651 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdaeb21-2420-46a9-aa0c-77058e61d137 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:38.287955 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdaeb21-2420-46a9-aa0c-77058e61d137 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:38.306044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e52b8a-40e6-4804-a608-9cccf43ed1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:38.306699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52b8a-40e6-4804-a608-9cccf43ed1d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:38.320294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52b8a-40e6-4804-a608-9cccf43ed1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:38.320636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e52b8a-40e6-4804-a608-9cccf43ed1d1 admin admin] PATH_INFO: `/v3/users/60201ffe2dcf460d8d643e8e0b749c03` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:38.328993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e52b8a-40e6-4804-a608-9cccf43ed1d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=60201ffe2dcf460d8d643e8e0b749c03)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:38.335458 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e52b8a-40e6-4804-a608-9cccf43ed1d1 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:38.388315 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a5e52b8a-40e6-4804-a608-9cccf43ed1d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60201ffe2dcf460d8d643e8e0b749c03'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:38.396085 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a5e52b8a-40e6-4804-a608-9cccf43ed1d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60201ffe2dcf460d8d643e8e0b749c03'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:38.408689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a5e52b8a-40e6-4804-a608-9cccf43ed1d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60201ffe2dcf460d8d643e8e0b749c03'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:38.409422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a5e52b8a-40e6-4804-a608-9cccf43ed1d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60201ffe2dcf460d8d643e8e0b749c03'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:38.410095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-a5e52b8a-40e6-4804-a608-9cccf43ed1d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60201ffe2dcf460d8d643e8e0b749c03'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:38.411928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:INSERT=1,keystone:SELECT=90,keystone:DELETE=9 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:38.428045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1210/2433] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:15:38 2022] DELETE /identity/v3/users/60201ffe2dcf460d8d643e8e0b749c03 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:38.433572 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c47e52-7db5-45f3-a311-ba72a24ee23c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:38.434086 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c47e52-7db5-45f3-a311-ba72a24ee23c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:38.434179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c47e52-7db5-45f3-a311-ba72a24ee23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:38.454470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-63c47e52-7db5-45f3-a311-ba72a24ee23c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:38.455671 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-63c47e52-7db5-45f3-a311-ba72a24ee23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:38.555313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1224/2434] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:38.566403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e52b8a-40e6-4804-a608-9cccf43ed1d1 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:38.568537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a5e52b8a-40e6-4804-a608-9cccf43ed1d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:38.636170 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e52b8a-40e6-4804-a608-9cccf43ed1d1 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:38.639917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8216d8-9de0-4a9f-bb73-19650df4b83f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:38.640595 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8216d8-9de0-4a9f-bb73-19650df4b83f service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:38.640944 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8216d8-9de0-4a9f-bb73-19650df4b83f service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:38.641319 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8216d8-9de0-4a9f-bb73-19650df4b83f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:38.654544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8216d8-9de0-4a9f-bb73-19650df4b83f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:38.656068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8216d8-9de0-4a9f-bb73-19650df4b83f service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:38.667290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1211/2435] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:38.950851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c47e52-7db5-45f3-a311-ba72a24ee23c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:38.952411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-63c47e52-7db5-45f3-a311-ba72a24ee23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:38.966690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c47e52-7db5-45f3-a311-ba72a24ee23c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:38.972329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c98c50-94f0-45bb-8cca-2fd94b2c5bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:38.973040 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c98c50-94f0-45bb-8cca-2fd94b2c5bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:38.973427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c98c50-94f0-45bb-8cca-2fd94b2c5bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:38.973780 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c98c50-94f0-45bb-8cca-2fd94b2c5bd7 admin admin] PATH_INFO: `/v3/projects/2713c78885694217b7869d91f83f0e7c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:38.975015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c98c50-94f0-45bb-8cca-2fd94b2c5bd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2713c78885694217b7869d91f83f0e7c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:38.977071 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c98c50-94f0-45bb-8cca-2fd94b2c5bd7 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:39.032043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e8c98c50-94f0-45bb-8cca-2fd94b2c5bd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2713c78885694217b7869d91f83f0e7c', 'request_id': 'req-e8c98c50-94f0-45bb-8cca-2fd94b2c5bd7'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:39.038365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e8c98c50-94f0-45bb-8cca-2fd94b2c5bd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2713c78885694217b7869d91f83f0e7c', 'request_id': 'req-e8c98c50-94f0-45bb-8cca-2fd94b2c5bd7'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:39.044499 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e8c98c50-94f0-45bb-8cca-2fd94b2c5bd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2713c78885694217b7869d91f83f0e7c', 'request_id': 'req-e8c98c50-94f0-45bb-8cca-2fd94b2c5bd7'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:39.046322 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:39.057918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e8c98c50-94f0-45bb-8cca-2fd94b2c5bd7 admin admin] The token cache is being invalidate because project 2713c78885694217b7869d91f83f0e7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:15:39.058204 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e8c98c50-94f0-45bb-8cca-2fd94b2c5bd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:39.058898 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e8c98c50-94f0-45bb-8cca-2fd94b2c5bd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:39.060479 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1225/2436] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:15:38 2022] DELETE /identity/v3/projects/2713c78885694217b7869d91f83f0e7c => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:39.065281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ffb14-8c0b-46f5-ab7d-793c6e9d1c62 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:39.065577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ffb14-8c0b-46f5-ab7d-793c6e9d1c62 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:39.065851 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750ffb14-8c0b-46f5-ab7d-793c6e9d1c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:39.085795 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-750ffb14-8c0b-46f5-ab7d-793c6e9d1c62 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:39.086993 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-750ffb14-8c0b-46f5-ab7d-793c6e9d1c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:39.146222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1212/2437] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:39.154701 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd84fa6-39bd-41d0-8258-67a9c16934b0 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:39.154983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd84fa6-39bd-41d0-8258-67a9c16934b0 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:39.155251 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd84fa6-39bd-41d0-8258-67a9c16934b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:39.176543 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ebd84fa6-39bd-41d0-8258-67a9c16934b0 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:39.177919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ebd84fa6-39bd-41d0-8258-67a9c16934b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:39.189312 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1226/2438] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:39.197439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ffb14-8c0b-46f5-ab7d-793c6e9d1c62 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:39.207975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750ffb14-8c0b-46f5-ab7d-793c6e9d1c62 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:39.213142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc9194-778e-4218-95c9-b733bc88fcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:39.214481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc9194-778e-4218-95c9-b733bc88fcdf admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:39.214758 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc9194-778e-4218-95c9-b733bc88fcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:39.215025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc9194-778e-4218-95c9-b733bc88fcdf admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:39.215667 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc9194-778e-4218-95c9-b733bc88fcdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:15:39.215961 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc9194-778e-4218-95c9-b733bc88fcdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:39.217587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc9194-778e-4218-95c9-b733bc88fcdf admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:39.235304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1213/2439] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:15:39.239819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd84fa6-39bd-41d0-8258-67a9c16934b0 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:39.249701 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd84fa6-39bd-41d0-8258-67a9c16934b0 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:39.256181 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9b00b-0d6a-40fc-9b35-b9d5e418fd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:39.256760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9b00b-0d6a-40fc-9b35-b9d5e418fd29 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:39.257040 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9b00b-0d6a-40fc-9b35-b9d5e418fd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:39.257361 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9b00b-0d6a-40fc-9b35-b9d5e418fd29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:39.258650 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9b00b-0d6a-40fc-9b35-b9d5e418fd29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:39.261112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9b00b-0d6a-40fc-9b35-b9d5e418fd29 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:39.289431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1227/2440] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:15:39 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:15:39.293825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc9194-778e-4218-95c9-b733bc88fcdf admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:39.304071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc9194-778e-4218-95c9-b733bc88fcdf admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:39.344775 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9b00b-0d6a-40fc-9b35-b9d5e418fd29 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:39.349442 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-caa9b00b-0d6a-40fc-9b35-b9d5e418fd29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:39.366660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7db82-4b17-4537-97f1-1f224ab58f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:39.367281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7db82-4b17-4537-97f1-1f224ab58f10 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:39.367553 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7db82-4b17-4537-97f1-1f224ab58f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:39.367816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7db82-4b17-4537-97f1-1f224ab58f10 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:39.368709 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7db82-4b17-4537-97f1-1f224ab58f10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:39.371508 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7db82-4b17-4537-97f1-1f224ab58f10 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:39.401252 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1214/2441] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:15:39 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:15:39.414089 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7db82-4b17-4537-97f1-1f224ab58f10 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:39.425772 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7db82-4b17-4537-97f1-1f224ab58f10 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:39.426576 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9b00b-0d6a-40fc-9b35-b9d5e418fd29 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:39.431335 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c63deb-f52e-438b-8a79-29b1cb51f89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:39.431599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c74d0a-b531-4760-88ed-cd43021df57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:39.432483 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c63deb-f52e-438b-8a79-29b1cb51f89a admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:39.432756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c63deb-f52e-438b-8a79-29b1cb51f89a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:39.432860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c74d0a-b531-4760-88ed-cd43021df57a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:39.433025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c63deb-f52e-438b-8a79-29b1cb51f89a admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:39.433152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c74d0a-b531-4760-88ed-cd43021df57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:39.433350 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c74d0a-b531-4760-88ed-cd43021df57a admin admin] PATH_INFO: `/v3/users/12f8acad04b04f499d40606a243fec5b` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:39.433689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c63deb-f52e-438b-8a79-29b1cb51f89a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:39.434784 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c74d0a-b531-4760-88ed-cd43021df57a admin admin] RBAC: Authorizing `identity:delete_user(user_id=12f8acad04b04f499d40606a243fec5b)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:39.435439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c63deb-f52e-438b-8a79-29b1cb51f89a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:39.437407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c74d0a-b531-4760-88ed-cd43021df57a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:39.441929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:39.449668 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1215/2442] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:15:39.453433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c63deb-f52e-438b-8a79-29b1cb51f89a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:39.470446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c63deb-f52e-438b-8a79-29b1cb51f89a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:39.476149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406c7cb-ad10-412b-8aa6-6e2b08667bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:39.476859 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406c7cb-ad10-412b-8aa6-6e2b08667bc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:39.477286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406c7cb-ad10-412b-8aa6-6e2b08667bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:39.477670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406c7cb-ad10-412b-8aa6-6e2b08667bc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:39.478422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406c7cb-ad10-412b-8aa6-6e2b08667bc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:39.480396 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b406c7cb-ad10-412b-8aa6-6e2b08667bc7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:39.491316 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1216/2443] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:15:39.496192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406c7cb-ad10-412b-8aa6-6e2b08667bc7 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:39.499680 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08c74d0a-b531-4760-88ed-cd43021df57a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12f8acad04b04f499d40606a243fec5b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:39.507660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406c7cb-ad10-412b-8aa6-6e2b08667bc7 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:39.509295 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08c74d0a-b531-4760-88ed-cd43021df57a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12f8acad04b04f499d40606a243fec5b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:39.513880 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efc562d-39c3-4bb8-9efd-97d582495f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:39.514618 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc562d-39c3-4bb8-9efd-97d582495f4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:39.514963 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc562d-39c3-4bb8-9efd-97d582495f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:39.515369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efc562d-39c3-4bb8-9efd-97d582495f4e admin admin] PATH_INFO: `/v3/projects/e6e7499326f34996a62da00eb5e21705/users/93aabb087e424130bf7ece089a575e2d/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:39.519418 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08c74d0a-b531-4760-88ed-cd43021df57a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12f8acad04b04f499d40606a243fec5b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:39.520159 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08c74d0a-b531-4760-88ed-cd43021df57a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12f8acad04b04f499d40606a243fec5b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:39.533447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc562d-39c3-4bb8-9efd-97d582495f4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6e7499326f34996a62da00eb5e21705, user_id=93aabb087e424130bf7ece089a575e2d, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:39.536413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efc562d-39c3-4bb8-9efd-97d582495f4e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:39.537545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08c74d0a-b531-4760-88ed-cd43021df57a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12f8acad04b04f499d40606a243fec5b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:39.548954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1228/2444] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:15:39 2022] DELETE /identity/v3/users/12f8acad04b04f499d40606a243fec5b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:39.550956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b8ade-326f-4e19-a369-fbe8d64a2dc6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:39.558072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b8ade-326f-4e19-a369-fbe8d64a2dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:39.558234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1217/2445] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:15:39 2022] PUT /identity/v3/projects/e6e7499326f34996a62da00eb5e21705/users/93aabb087e424130bf7ece089a575e2d/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:15:39.558332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b8ade-326f-4e19-a369-fbe8d64a2dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:39.563746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aa439-86e9-4ae0-9ade-637d0ee63cdd None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:39.564263 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aa439-86e9-4ae0-9ade-637d0ee63cdd None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:39.564606 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888aa439-86e9-4ae0-9ade-637d0ee63cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:39.576509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-3b7b8ade-326f-4e19-a369-fbe8d64a2dc6 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:39.577823 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3b7b8ade-326f-4e19-a369-fbe8d64a2dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:39.592741 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-888aa439-86e9-4ae0-9ade-637d0ee63cdd None None] MFA Rules not processed for user `93aabb087e424130bf7ece089a575e2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:39.593951 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-888aa439-86e9-4ae0-9ade-637d0ee63cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:39.617045 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:INSERT=3,keystone:DELETE=11,keystone:SELECT=86 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:39.642319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1229/2446] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:39.652700 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b8ade-326f-4e19-a369-fbe8d64a2dc6 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:39.654144 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3b7b8ade-326f-4e19-a369-fbe8d64a2dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:39.662872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1218/2447] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:39 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:15:39.674550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888aa439-86e9-4ae0-9ade-637d0ee63cdd None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:39.676053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-888aa439-86e9-4ae0-9ade-637d0ee63cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:39.713888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b8ade-326f-4e19-a369-fbe8d64a2dc6 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:39.717865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7fd69-0f33-4fd0-a8a3-9b2f969ffefc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:39.718905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7fd69-0f33-4fd0-a8a3-9b2f969ffefc service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:39.718905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7fd69-0f33-4fd0-a8a3-9b2f969ffefc service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:39.719147 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e7fd69-0f33-4fd0-a8a3-9b2f969ffefc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:39.728375 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7fd69-0f33-4fd0-a8a3-9b2f969ffefc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:39.730703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e7fd69-0f33-4fd0-a8a3-9b2f969ffefc service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:39.740547 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1230/2448] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:15:39.746491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888aa439-86e9-4ae0-9ade-637d0ee63cdd None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:39.750979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05683b71-5479-4cda-a611-03bd89dcdfd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:39.751699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05683b71-5479-4cda-a611-03bd89dcdfd3 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:39.752077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05683b71-5479-4cda-a611-03bd89dcdfd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:39.752444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05683b71-5479-4cda-a611-03bd89dcdfd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:39.769360 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05683b71-5479-4cda-a611-03bd89dcdfd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:39.771733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05683b71-5479-4cda-a611-03bd89dcdfd3 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:39.784447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1219/2449] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:39 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:15:40.089392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7fd69-0f33-4fd0-a8a3-9b2f969ffefc service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:40.090922 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-50e7fd69-0f33-4fd0-a8a3-9b2f969ffefc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:40.109252 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e7fd69-0f33-4fd0-a8a3-9b2f969ffefc service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:40.125143 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc28a7f-f816-4a3c-9352-65bc5bb1e079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:40.125897 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc28a7f-f816-4a3c-9352-65bc5bb1e079 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:40.126179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc28a7f-f816-4a3c-9352-65bc5bb1e079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:40.126456 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc28a7f-f816-4a3c-9352-65bc5bb1e079 admin admin] PATH_INFO: `/v3/projects/5a5946d4b3564b439464a90ad9334bb5` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:40.127770 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc28a7f-f816-4a3c-9352-65bc5bb1e079 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a5946d4b3564b439464a90ad9334bb5)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:40.129880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc28a7f-f816-4a3c-9352-65bc5bb1e079 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:40.212685 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ffc28a7f-f816-4a3c-9352-65bc5bb1e079 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a5946d4b3564b439464a90ad9334bb5', 'request_id': 'req-ffc28a7f-f816-4a3c-9352-65bc5bb1e079'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:40.222659 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ffc28a7f-f816-4a3c-9352-65bc5bb1e079 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a5946d4b3564b439464a90ad9334bb5', 'request_id': 'req-ffc28a7f-f816-4a3c-9352-65bc5bb1e079'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:40.230741 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ffc28a7f-f816-4a3c-9352-65bc5bb1e079 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a5946d4b3564b439464a90ad9334bb5', 'request_id': 'req-ffc28a7f-f816-4a3c-9352-65bc5bb1e079'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:40.263180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ffc28a7f-f816-4a3c-9352-65bc5bb1e079 admin admin] The token cache is being invalidate because project 5a5946d4b3564b439464a90ad9334bb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:15:40.265620 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ffc28a7f-f816-4a3c-9352-65bc5bb1e079 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:40.269807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ffc28a7f-f816-4a3c-9352-65bc5bb1e079 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:40.276328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1231/2450] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:15:40 2022] DELETE /identity/v3/projects/5a5946d4b3564b439464a90ad9334bb5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:40.278079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb334da-d11a-4597-9d6e-24b7989943ab None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:40.278937 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb334da-d11a-4597-9d6e-24b7989943ab None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:40.279227 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb334da-d11a-4597-9d6e-24b7989943ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:40.313617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-dcb334da-d11a-4597-9d6e-24b7989943ab None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:40.315003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-dcb334da-d11a-4597-9d6e-24b7989943ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:40.376255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:40.386847 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1220/2451] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:40.394931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b91cb-ccb6-4187-9d41-de25778aa7af None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:40.395343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b91cb-ccb6-4187-9d41-de25778aa7af None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:40.395713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624b91cb-ccb6-4187-9d41-de25778aa7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:40.417353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-624b91cb-ccb6-4187-9d41-de25778aa7af None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:40.418625 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-624b91cb-ccb6-4187-9d41-de25778aa7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:40.427380 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1232/2452] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:40.431473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb334da-d11a-4597-9d6e-24b7989943ab None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:40.442313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb334da-d11a-4597-9d6e-24b7989943ab None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:40.449628 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e334066-ecf3-4da3-8a20-8478f019999b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:40.450194 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e334066-ecf3-4da3-8a20-8478f019999b admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:40.450506 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e334066-ecf3-4da3-8a20-8478f019999b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:40.450761 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e334066-ecf3-4da3-8a20-8478f019999b admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:40.451728 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e334066-ecf3-4da3-8a20-8478f019999b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:15:40.452043 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e334066-ecf3-4da3-8a20-8478f019999b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:40.454754 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e334066-ecf3-4da3-8a20-8478f019999b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:40.477894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1221/2453] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:15:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:15:40.479303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b91cb-ccb6-4187-9d41-de25778aa7af None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:40.489267 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624b91cb-ccb6-4187-9d41-de25778aa7af None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:40.496257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b31ab9-73d6-47d5-9fac-417b3429841c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:40.496912 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b31ab9-73d6-47d5-9fac-417b3429841c admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:40.497212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b31ab9-73d6-47d5-9fac-417b3429841c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:40.497560 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b31ab9-73d6-47d5-9fac-417b3429841c admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:40.498193 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b31ab9-73d6-47d5-9fac-417b3429841c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:40.500170 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b31ab9-73d6-47d5-9fac-417b3429841c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:40.554808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1233/2454] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:15:40 2022] POST /identity/v3/projects => generated 381 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:15:40.559190 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e334066-ecf3-4da3-8a20-8478f019999b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:40.572225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e334066-ecf3-4da3-8a20-8478f019999b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:40.648347 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc27035-6a9b-45bf-a184-99fed86523e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:40.649992 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc27035-6a9b-45bf-a184-99fed86523e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:40.650313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc27035-6a9b-45bf-a184-99fed86523e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:40.650561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc27035-6a9b-45bf-a184-99fed86523e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:40.652765 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc27035-6a9b-45bf-a184-99fed86523e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:40.656745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc27035-6a9b-45bf-a184-99fed86523e5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:40.682882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1222/2455] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:15:40 2022] POST /identity/v3/users => generated 291 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:15:40.687495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b31ab9-73d6-47d5-9fac-417b3429841c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:40.709695 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b31ab9-73d6-47d5-9fac-417b3429841c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:40.718033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca9fd6-fb47-41f8-96b2-2752aaf493ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:40.718742 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca9fd6-fb47-41f8-96b2-2752aaf493ad admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:40.719083 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca9fd6-fb47-41f8-96b2-2752aaf493ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:40.719418 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca9fd6-fb47-41f8-96b2-2752aaf493ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:40.720127 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca9fd6-fb47-41f8-96b2-2752aaf493ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:40.721894 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca9fd6-fb47-41f8-96b2-2752aaf493ad admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:40.733514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1234/2456] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:15:40.737494 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc27035-6a9b-45bf-a184-99fed86523e5 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:40.747529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc27035-6a9b-45bf-a184-99fed86523e5 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:40.753105 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ce9fc2-74ae-452f-b7f8-ab7404054a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:40.753910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce9fc2-74ae-452f-b7f8-ab7404054a19 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:40.754292 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce9fc2-74ae-452f-b7f8-ab7404054a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:40.754568 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ce9fc2-74ae-452f-b7f8-ab7404054a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:40.755234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce9fc2-74ae-452f-b7f8-ab7404054a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:40.756821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ce9fc2-74ae-452f-b7f8-ab7404054a19 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:40.772939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1223/2457] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:15:40.774225 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca9fd6-fb47-41f8-96b2-2752aaf493ad admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:40.784712 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca9fd6-fb47-41f8-96b2-2752aaf493ad admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:40.790196 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6b0ac-3097-4560-9dc9-6366aeac4071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:40.790937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6b0ac-3097-4560-9dc9-6366aeac4071 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:40.791365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6b0ac-3097-4560-9dc9-6366aeac4071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:40.791725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6b0ac-3097-4560-9dc9-6366aeac4071 admin admin] PATH_INFO: `/v3/projects/08d5f61674804f93bd2127fc76919bd3/users/b1b8da543ad543bb95881450e4f9b585/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:40.809897 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6b0ac-3097-4560-9dc9-6366aeac4071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08d5f61674804f93bd2127fc76919bd3, user_id=b1b8da543ad543bb95881450e4f9b585, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:40.812780 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6b0ac-3097-4560-9dc9-6366aeac4071 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:40.828648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1235/2458] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:15:40 2022] PUT /identity/v3/projects/08d5f61674804f93bd2127fc76919bd3/users/b1b8da543ad543bb95881450e4f9b585/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:40.833696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189ad8f-f9de-4c4c-bd47-5f035b1c9178 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:40.834084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189ad8f-f9de-4c4c-bd47-5f035b1c9178 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:40.834483 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189ad8f-f9de-4c4c-bd47-5f035b1c9178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:40.856212 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2189ad8f-f9de-4c4c-bd47-5f035b1c9178 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:40.857461 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2189ad8f-f9de-4c4c-bd47-5f035b1c9178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:40.958383 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1224/2459] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:40.968442 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6b0ac-3097-4560-9dc9-6366aeac4071 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:40.970560 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bfb6b0ac-3097-4560-9dc9-6366aeac4071 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:40.977551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:41.034382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6b0ac-3097-4560-9dc9-6366aeac4071 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:41.037977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e16410-085e-45e1-bd7d-c84f5f9e542c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:41.038614 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e16410-085e-45e1-bd7d-c84f5f9e542c service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:41.038889 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e16410-085e-45e1-bd7d-c84f5f9e542c service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:41.039160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e16410-085e-45e1-bd7d-c84f5f9e542c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:41.048977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e16410-085e-45e1-bd7d-c84f5f9e542c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:41.050417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e16410-085e-45e1-bd7d-c84f5f9e542c service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:41.060061 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1236/2460] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:41.285270 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189ad8f-f9de-4c4c-bd47-5f035b1c9178 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:41.287404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2189ad8f-f9de-4c4c-bd47-5f035b1c9178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:41.300244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189ad8f-f9de-4c4c-bd47-5f035b1c9178 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:41.305393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d45198-0a37-46cc-955b-7e1dca77269c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:41.306031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d45198-0a37-46cc-955b-7e1dca77269c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:41.306423 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d45198-0a37-46cc-955b-7e1dca77269c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:41.306726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d45198-0a37-46cc-955b-7e1dca77269c admin admin] PATH_INFO: `/v3/users/0da78d6f291c438baa9d1fa47ded4c8e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:41.307920 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d45198-0a37-46cc-955b-7e1dca77269c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0da78d6f291c438baa9d1fa47ded4c8e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:41.309851 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d45198-0a37-46cc-955b-7e1dca77269c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:41.360387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c3d45198-0a37-46cc-955b-7e1dca77269c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0da78d6f291c438baa9d1fa47ded4c8e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:41.370086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c3d45198-0a37-46cc-955b-7e1dca77269c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0da78d6f291c438baa9d1fa47ded4c8e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:41.372968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:41.420160 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c3d45198-0a37-46cc-955b-7e1dca77269c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0da78d6f291c438baa9d1fa47ded4c8e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:41.421354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c3d45198-0a37-46cc-955b-7e1dca77269c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0da78d6f291c438baa9d1fa47ded4c8e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:41.422067 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c3d45198-0a37-46cc-955b-7e1dca77269c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0da78d6f291c438baa9d1fa47ded4c8e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:41.444245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1225/2461] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:15:41 2022] DELETE /identity/v3/users/0da78d6f291c438baa9d1fa47ded4c8e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:41.445567 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5beaf25-6bf5-451b-b6ec-310fce70a1b6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:41.446109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5beaf25-6bf5-451b-b6ec-310fce70a1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:41.446634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5beaf25-6bf5-451b-b6ec-310fce70a1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:41.474434 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b5beaf25-6bf5-451b-b6ec-310fce70a1b6 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:41.475664 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b5beaf25-6bf5-451b-b6ec-310fce70a1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:41.562581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1237/2462] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:15:41.572608 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d45198-0a37-46cc-955b-7e1dca77269c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:41.574571 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c3d45198-0a37-46cc-955b-7e1dca77269c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:41.635037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d45198-0a37-46cc-955b-7e1dca77269c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:41.639080 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46c286-812a-4549-b4b5-81cfa379d83a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:41.639774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46c286-812a-4549-b4b5-81cfa379d83a service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:41.640124 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46c286-812a-4549-b4b5-81cfa379d83a service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:41.640449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46c286-812a-4549-b4b5-81cfa379d83a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:41.650153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46c286-812a-4549-b4b5-81cfa379d83a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:41.651782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46c286-812a-4549-b4b5-81cfa379d83a service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:41.662162 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1226/2463] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:41.783392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb11db-2bec-43db-825a-28d6319d6a14 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:41.783746 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb11db-2bec-43db-825a-28d6319d6a14 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:41.784000 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb11db-2bec-43db-825a-28d6319d6a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:41.814581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0edb11db-2bec-43db-825a-28d6319d6a14 None None] MFA Rules not processed for user `b1b8da543ad543bb95881450e4f9b585`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:41.815824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0edb11db-2bec-43db-825a-28d6319d6a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:41.916840 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1238/2464] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:41.927550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46c286-812a-4549-b4b5-81cfa379d83a service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:41.929047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cb46c286-812a-4549-b4b5-81cfa379d83a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:41.995118 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46c286-812a-4549-b4b5-81cfa379d83a service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:41.999146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bfdb2f-da75-4556-b389-266aafd3b8e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:41.999764 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfdb2f-da75-4556-b389-266aafd3b8e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:42.000044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfdb2f-da75-4556-b389-266aafd3b8e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:42.000316 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bfdb2f-da75-4556-b389-266aafd3b8e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:42.009794 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfdb2f-da75-4556-b389-266aafd3b8e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:42.011454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bfdb2f-da75-4556-b389-266aafd3b8e5 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:42.021393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1227/2465] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:41 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:42.214692 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb11db-2bec-43db-825a-28d6319d6a14 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:42.216180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0edb11db-2bec-43db-825a-28d6319d6a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:42.242473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb11db-2bec-43db-825a-28d6319d6a14 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:42.256021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b02f56-8f27-4f13-bc13-d614e79db982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:42.257336 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b02f56-8f27-4f13-bc13-d614e79db982 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:42.257869 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b02f56-8f27-4f13-bc13-d614e79db982 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:42.257869 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b02f56-8f27-4f13-bc13-d614e79db982 admin admin] PATH_INFO: `/v3/projects/e77fcc563adb43d9b38c5a944969bf2d` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:42.259120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b02f56-8f27-4f13-bc13-d614e79db982 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e77fcc563adb43d9b38c5a944969bf2d)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:42.261270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b02f56-8f27-4f13-bc13-d614e79db982 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:42.325733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:42.363844 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08b02f56-8f27-4f13-bc13-d614e79db982 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e77fcc563adb43d9b38c5a944969bf2d', 'request_id': 'req-08b02f56-8f27-4f13-bc13-d614e79db982'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:42.371944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08b02f56-8f27-4f13-bc13-d614e79db982 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e77fcc563adb43d9b38c5a944969bf2d', 'request_id': 'req-08b02f56-8f27-4f13-bc13-d614e79db982'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:42.382820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08b02f56-8f27-4f13-bc13-d614e79db982 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e77fcc563adb43d9b38c5a944969bf2d', 'request_id': 'req-08b02f56-8f27-4f13-bc13-d614e79db982'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:42.407029 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08b02f56-8f27-4f13-bc13-d614e79db982 admin admin] The token cache is being invalidate because project e77fcc563adb43d9b38c5a944969bf2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:15:42.407331 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08b02f56-8f27-4f13-bc13-d614e79db982 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:42.408005 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-08b02f56-8f27-4f13-bc13-d614e79db982 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:15:42.409742 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1239/2466] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:15:42 2022] DELETE /identity/v3/projects/e77fcc563adb43d9b38c5a944969bf2d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:42.414554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950ef68-1ca0-482c-98c2-77620e74af42 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:42.415060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950ef68-1ca0-482c-98c2-77620e74af42 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:42.415156 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9950ef68-1ca0-482c-98c2-77620e74af42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:42.441946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9950ef68-1ca0-482c-98c2-77620e74af42 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:42.443188 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9950ef68-1ca0-482c-98c2-77620e74af42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:42.536617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1228/2467] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:42.546242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7e55c-1106-4b01-8d37-6169b8d242df None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:42.546242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7e55c-1106-4b01-8d37-6169b8d242df None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:42.546242 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de7e55c-1106-4b01-8d37-6169b8d242df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:42.569087 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-3de7e55c-1106-4b01-8d37-6169b8d242df None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:42.570311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3de7e55c-1106-4b01-8d37-6169b8d242df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:42.583728 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1240/2468] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:42.588577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950ef68-1ca0-482c-98c2-77620e74af42 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:42.598292 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9950ef68-1ca0-482c-98c2-77620e74af42 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:42.604828 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff91744-6883-42c4-8da8-a26e5d79e1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:42.605736 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff91744-6883-42c4-8da8-a26e5d79e1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:42.606038 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff91744-6883-42c4-8da8-a26e5d79e1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:42.606335 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff91744-6883-42c4-8da8-a26e5d79e1a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:42.607115 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff91744-6883-42c4-8da8-a26e5d79e1a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:15:42.607419 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff91744-6883-42c4-8da8-a26e5d79e1a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:42.609050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff91744-6883-42c4-8da8-a26e5d79e1a9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:42.621226 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1229/2469] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:15:42.625677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7e55c-1106-4b01-8d37-6169b8d242df None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:42.636706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de7e55c-1106-4b01-8d37-6169b8d242df None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:42.641435 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424e7bd-b6b0-41f7-b669-53acc4bfba20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:42.642039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424e7bd-b6b0-41f7-b669-53acc4bfba20 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:42.642510 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424e7bd-b6b0-41f7-b669-53acc4bfba20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:42.642797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6424e7bd-b6b0-41f7-b669-53acc4bfba20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:42.643553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424e7bd-b6b0-41f7-b669-53acc4bfba20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:42.645348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6424e7bd-b6b0-41f7-b669-53acc4bfba20 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:42.691248 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1241/2470] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:15:42 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:15:42.692341 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff91744-6883-42c4-8da8-a26e5d79e1a9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:42.703120 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff91744-6883-42c4-8da8-a26e5d79e1a9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:42.708540 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:42.776549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac4c02-82ae-4f2e-93e2-d2f19714961c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:42.777158 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac4c02-82ae-4f2e-93e2-d2f19714961c admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:42.777489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac4c02-82ae-4f2e-93e2-d2f19714961c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:42.777744 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ac4c02-82ae-4f2e-93e2-d2f19714961c admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:42.778326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac4c02-82ae-4f2e-93e2-d2f19714961c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:42.780220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ac4c02-82ae-4f2e-93e2-d2f19714961c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:42.806332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1230/2471] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:15:42 2022] POST /identity/v3/users => generated 293 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:15:42.810719 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424e7bd-b6b0-41f7-b669-53acc4bfba20 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:42.820780 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6424e7bd-b6b0-41f7-b669-53acc4bfba20 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:42.825986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd93bbd-f208-4a6a-8a0d-2dc51ec239ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:42.826588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd93bbd-f208-4a6a-8a0d-2dc51ec239ae admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:42.826869 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd93bbd-f208-4a6a-8a0d-2dc51ec239ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:42.827143 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd93bbd-f208-4a6a-8a0d-2dc51ec239ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:42.827810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd93bbd-f208-4a6a-8a0d-2dc51ec239ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:42.829432 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd93bbd-f208-4a6a-8a0d-2dc51ec239ae admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:42.847911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1242/2472] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:15:42.850275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac4c02-82ae-4f2e-93e2-d2f19714961c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:42.862250 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ac4c02-82ae-4f2e-93e2-d2f19714961c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:42.866921 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140497ab-f3d6-4d42-bc12-582371ae8b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:42.867525 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140497ab-f3d6-4d42-bc12-582371ae8b68 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:42.867802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140497ab-f3d6-4d42-bc12-582371ae8b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:42.868068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140497ab-f3d6-4d42-bc12-582371ae8b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:42.868698 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140497ab-f3d6-4d42-bc12-582371ae8b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:42.870359 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140497ab-f3d6-4d42-bc12-582371ae8b68 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:42.882713 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1231/2473] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:15:42.887114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd93bbd-f208-4a6a-8a0d-2dc51ec239ae admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:42.900370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd93bbd-f208-4a6a-8a0d-2dc51ec239ae admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:42.907730 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fefea82-930e-4ea3-8582-5095fa91d4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:42.908412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefea82-930e-4ea3-8582-5095fa91d4ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:42.908628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefea82-930e-4ea3-8582-5095fa91d4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:42.908873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fefea82-930e-4ea3-8582-5095fa91d4ae admin admin] PATH_INFO: `/v3/projects/7450d4b1a8fe4762b7f8cbfaf7163bce/users/0c4a21ea6a2c4870a2fe009027c12f9d/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:42.924088 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fefea82-930e-4ea3-8582-5095fa91d4ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=7450d4b1a8fe4762b7f8cbfaf7163bce, user_id=0c4a21ea6a2c4870a2fe009027c12f9d, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:42.926836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fefea82-930e-4ea3-8582-5095fa91d4ae admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:42.939620 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1243/2474] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:15:42 2022] PUT /identity/v3/projects/7450d4b1a8fe4762b7f8cbfaf7163bce/users/0c4a21ea6a2c4870a2fe009027c12f9d/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:15:42.950224 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71668cb-0f7f-40c8-8d29-6cb9c15e6ec1 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:42.950507 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71668cb-0f7f-40c8-8d29-6cb9c15e6ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:42.950780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71668cb-0f7f-40c8-8d29-6cb9c15e6ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:42.979695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a71668cb-0f7f-40c8-8d29-6cb9c15e6ec1 None None] MFA Rules not processed for user `0c4a21ea6a2c4870a2fe009027c12f9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:42.981168 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a71668cb-0f7f-40c8-8d29-6cb9c15e6ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:43.064938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1232/2475] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:43.076420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fefea82-930e-4ea3-8582-5095fa91d4ae admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:43.078793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4fefea82-930e-4ea3-8582-5095fa91d4ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:43.166705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fefea82-930e-4ea3-8582-5095fa91d4ae admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:43.172557 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d2d0a2-4a37-4bcc-8874-961c8fe7000f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:43.173207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2d0a2-4a37-4bcc-8874-961c8fe7000f service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:43.173696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2d0a2-4a37-4bcc-8874-961c8fe7000f service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:43.173975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d2d0a2-4a37-4bcc-8874-961c8fe7000f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:43.183672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2d0a2-4a37-4bcc-8874-961c8fe7000f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:43.185408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d2d0a2-4a37-4bcc-8874-961c8fe7000f service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:43.194515 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1244/2476] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:15:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:15:43.485360 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b86eb-87d9-420e-9fdd-4ae1536145f8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:43.486378 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b86eb-87d9-420e-9fdd-4ae1536145f8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:43.486731 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172b86eb-87d9-420e-9fdd-4ae1536145f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:43.550580 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-172b86eb-87d9-420e-9fdd-4ae1536145f8 None None] MFA Rules not processed for user `b1b8da543ad543bb95881450e4f9b585`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:43.551936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-172b86eb-87d9-420e-9fdd-4ae1536145f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:43.665970 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1233/2477] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:43 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:44.616366 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20624d4f-c5a9-4e7f-9a46-4f3a3899fa6e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:44.616845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20624d4f-c5a9-4e7f-9a46-4f3a3899fa6e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:44.616952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20624d4f-c5a9-4e7f-9a46-4f3a3899fa6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:44.642480 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-20624d4f-c5a9-4e7f-9a46-4f3a3899fa6e None None] MFA Rules not processed for user `aa81a686e4324370a20e3fc35e5cc7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:44.643914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-20624d4f-c5a9-4e7f-9a46-4f3a3899fa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:44.709055 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1245/2478] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:15:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:49.407470 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44ad71-c94c-489e-8112-b97165e27e00 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:49.408484 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44ad71-c94c-489e-8112-b97165e27e00 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:49.408973 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44ad71-c94c-489e-8112-b97165e27e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:49.433688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-0e44ad71-c94c-489e-8112-b97165e27e00 None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:49.435496 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0e44ad71-c94c-489e-8112-b97165e27e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:49.456261 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:49.538703 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1234/2479] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 19:15:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:49.594726 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20624d4f-c5a9-4e7f-9a46-4f3a3899fa6e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:49.596760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-20624d4f-c5a9-4e7f-9a46-4f3a3899fa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:49.604428 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:15:49.737798 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20624d4f-c5a9-4e7f-9a46-4f3a3899fa6e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:49.755734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80f8f2-2aba-46f9-926d-82f38beab864 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:49.756668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80f8f2-2aba-46f9-926d-82f38beab864 service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:49.757047 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80f8f2-2aba-46f9-926d-82f38beab864 service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:49.757539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d80f8f2-2aba-46f9-926d-82f38beab864 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:49.769827 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80f8f2-2aba-46f9-926d-82f38beab864 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:49.772057 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d80f8f2-2aba-46f9-926d-82f38beab864 service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:49.783873 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1246/2480] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:15:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:15:50.353550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdbf38-7473-4dc3-8610-2042654d3f5f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:50.353860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdbf38-7473-4dc3-8610-2042654d3f5f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:50.354133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fdbf38-7473-4dc3-8610-2042654d3f5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:50.373541 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-29fdbf38-7473-4dc3-8610-2042654d3f5f None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:50.374727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-29fdbf38-7473-4dc3-8610-2042654d3f5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:50.382344 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1235/2481] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 31 19:15:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:50.396279 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80f8f2-2aba-46f9-926d-82f38beab864 service swift] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:50.407563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d80f8f2-2aba-46f9-926d-82f38beab864 service swift] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:50.412834 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6536e4-e1bd-421e-94b7-9d7c0d2fcdd6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:50.413527 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6536e4-e1bd-421e-94b7-9d7c0d2fcdd6 service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:50.413949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6536e4-e1bd-421e-94b7-9d7c0d2fcdd6 service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:50.414287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6536e4-e1bd-421e-94b7-9d7c0d2fcdd6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:50.423197 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6536e4-e1bd-421e-94b7-9d7c0d2fcdd6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:50.424903 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6536e4-e1bd-421e-94b7-9d7c0d2fcdd6 service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:50.433507 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1247/2482] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:15:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:15:50.666204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161444c-0809-4a17-bee8-13f52bf787b7 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:50.666686 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161444c-0809-4a17-bee8-13f52bf787b7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:50.667109 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c161444c-0809-4a17-bee8-13f52bf787b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:50.686604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c161444c-0809-4a17-bee8-13f52bf787b7 None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:50.687783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c161444c-0809-4a17-bee8-13f52bf787b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:50.695468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1236/2483] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 19:15:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:15:50.707821 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6536e4-e1bd-421e-94b7-9d7c0d2fcdd6 service swift] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:50.718246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6536e4-e1bd-421e-94b7-9d7c0d2fcdd6 service swift] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:50.723131 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be217b2-29bd-4da2-8ea6-f9a35c477fb8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:50.723713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be217b2-29bd-4da2-8ea6-f9a35c477fb8 service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:50.723990 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be217b2-29bd-4da2-8ea6-f9a35c477fb8 service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:50.724261 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be217b2-29bd-4da2-8ea6-f9a35c477fb8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:50.732294 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be217b2-29bd-4da2-8ea6-f9a35c477fb8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:50.733882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be217b2-29bd-4da2-8ea6-f9a35c477fb8 service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:50.741735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1248/2484] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:15:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:15:51.078670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550a184-841a-4678-b865-71779351fc99 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:51.078957 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550a184-841a-4678-b865-71779351fc99 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:51.079235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a550a184-841a-4678-b865-71779351fc99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:51.099579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a550a184-841a-4678-b865-71779351fc99 None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:15:51.100807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a550a184-841a-4678-b865-71779351fc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:15:51.108783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1237/2485] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 31 19:15:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:15:51.121359 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be217b2-29bd-4da2-8ea6-f9a35c477fb8 service swift] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:15:51.132391 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be217b2-29bd-4da2-8ea6-f9a35c477fb8 service swift] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:15:51.138544 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7eb95d-7120-452c-a339-cc9b823d87e7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:15:51.139144 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eb95d-7120-452c-a339-cc9b823d87e7 service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:15:51.139436 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eb95d-7120-452c-a339-cc9b823d87e7 service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:15:51.139681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7eb95d-7120-452c-a339-cc9b823d87e7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:15:51.148464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7eb95d-7120-452c-a339-cc9b823d87e7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:15:51.150304 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7eb95d-7120-452c-a339-cc9b823d87e7 service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:15:51.159094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1249/2486] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:15:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:16:01.094078 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=24 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:01.155713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=48 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:12.356001 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a550a184-841a-4678-b865-71779351fc99 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:12.357461 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a550a184-841a-4678-b865-71779351fc99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:12.426507 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a550a184-841a-4678-b865-71779351fc99 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:12.430919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9da57-8859-4c29-b46e-8cffaa3046f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:12.431590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9da57-8859-4c29-b46e-8cffaa3046f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:12.431863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9da57-8859-4c29-b46e-8cffaa3046f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:12.432127 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9da57-8859-4c29-b46e-8cffaa3046f9 admin admin] PATH_INFO: `/v3/users/426bb2c78bce40179a0a115651b81a5b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:12.433255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9da57-8859-4c29-b46e-8cffaa3046f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=426bb2c78bce40179a0a115651b81a5b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:12.435147 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9da57-8859-4c29-b46e-8cffaa3046f9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:12.484211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-34a9da57-8859-4c29-b46e-8cffaa3046f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '426bb2c78bce40179a0a115651b81a5b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:12.491016 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-34a9da57-8859-4c29-b46e-8cffaa3046f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '426bb2c78bce40179a0a115651b81a5b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:12.502710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-34a9da57-8859-4c29-b46e-8cffaa3046f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '426bb2c78bce40179a0a115651b81a5b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:12.503465 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-34a9da57-8859-4c29-b46e-8cffaa3046f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '426bb2c78bce40179a0a115651b81a5b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:12.504170 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-34a9da57-8859-4c29-b46e-8cffaa3046f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '426bb2c78bce40179a0a115651b81a5b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:12.516940 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1238/2487] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:16:12 2022] DELETE /identity/v3/users/426bb2c78bce40179a0a115651b81a5b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:16:12.521521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97db624-89ad-4dcb-a693-3037a61a9206 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:12.522054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97db624-89ad-4dcb-a693-3037a61a9206 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:12.522233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97db624-89ad-4dcb-a693-3037a61a9206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:12.541442 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b97db624-89ad-4dcb-a693-3037a61a9206 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:12.542636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b97db624-89ad-4dcb-a693-3037a61a9206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:12.601308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1250/2488] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:16:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:12.611285 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9da57-8859-4c29-b46e-8cffaa3046f9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:12.612689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-34a9da57-8859-4c29-b46e-8cffaa3046f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:12.670737 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9da57-8859-4c29-b46e-8cffaa3046f9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:12.674296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4337ef-0c3d-4e47-9591-01d5318e764c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:12.674889 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4337ef-0c3d-4e47-9591-01d5318e764c service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:12.675162 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4337ef-0c3d-4e47-9591-01d5318e764c service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:12.675430 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4337ef-0c3d-4e47-9591-01d5318e764c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:12.686184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4337ef-0c3d-4e47-9591-01d5318e764c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:12.687422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4337ef-0c3d-4e47-9591-01d5318e764c service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:12.696841 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1239/2489] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:16:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:16:12.932817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97db624-89ad-4dcb-a693-3037a61a9206 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:12.935166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b97db624-89ad-4dcb-a693-3037a61a9206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:12.954195 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97db624-89ad-4dcb-a693-3037a61a9206 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:12.972828 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12db00-b502-4107-babe-03264011944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:12.974057 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12db00-b502-4107-babe-03264011944d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:12.974943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12db00-b502-4107-babe-03264011944d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:12.974943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12db00-b502-4107-babe-03264011944d admin admin] PATH_INFO: `/v3/projects/f5e4dacbe64a48f49f697787961e1200` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:12.976097 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12db00-b502-4107-babe-03264011944d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5e4dacbe64a48f49f697787961e1200)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:12.979246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12db00-b502-4107-babe-03264011944d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:12.993531 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875dfba-7cbe-4c96-a2f0-0425e95b1fa4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:12.993835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875dfba-7cbe-4c96-a2f0-0425e95b1fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:12.994106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c875dfba-7cbe-4c96-a2f0-0425e95b1fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:13.027917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c875dfba-7cbe-4c96-a2f0-0425e95b1fa4 None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:13.029709 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c875dfba-7cbe-4c96-a2f0-0425e95b1fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:13.052142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dc12db00-b502-4107-babe-03264011944d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5e4dacbe64a48f49f697787961e1200', 'request_id': 'req-dc12db00-b502-4107-babe-03264011944d'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:13.060305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dc12db00-b502-4107-babe-03264011944d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5e4dacbe64a48f49f697787961e1200', 'request_id': 'req-dc12db00-b502-4107-babe-03264011944d'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:13.066618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dc12db00-b502-4107-babe-03264011944d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5e4dacbe64a48f49f697787961e1200', 'request_id': 'req-dc12db00-b502-4107-babe-03264011944d'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:13.068475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:13.083271 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dc12db00-b502-4107-babe-03264011944d admin admin] The token cache is being invalidate because project f5e4dacbe64a48f49f697787961e1200 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:16:13.083840 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dc12db00-b502-4107-babe-03264011944d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:13.085630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-dc12db00-b502-4107-babe-03264011944d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:13.087346 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1251/2490] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:16:12 2022] DELETE /identity/v3/projects/f5e4dacbe64a48f49f697787961e1200 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:16:13.092745 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e372d2-af9b-4210-8eb7-21ff3a55d185 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:13.093065 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e372d2-af9b-4210-8eb7-21ff3a55d185 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:13.093393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e372d2-af9b-4210-8eb7-21ff3a55d185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:13.121074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-69e372d2-af9b-4210-8eb7-21ff3a55d185 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:13.122689 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-69e372d2-af9b-4210-8eb7-21ff3a55d185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:13.124726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1240/2491] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 19:16:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:16:13.141122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875dfba-7cbe-4c96-a2f0-0425e95b1fa4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:13.144933 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c875dfba-7cbe-4c96-a2f0-0425e95b1fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:13.205337 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1252/2492] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:13.253960 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c875dfba-7cbe-4c96-a2f0-0425e95b1fa4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:13.266208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3488b-dd48-4e6e-8e83-0ba2b68f5414 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:13.266485 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3488b-dd48-4e6e-8e83-0ba2b68f5414 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:13.266755 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3488b-dd48-4e6e-8e83-0ba2b68f5414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:13.287338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-67a3488b-dd48-4e6e-8e83-0ba2b68f5414 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:13.288546 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-67a3488b-dd48-4e6e-8e83-0ba2b68f5414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:13.297163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1253/2493] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:16:13.301246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3488b-dd48-4e6e-8e83-0ba2b68f5414 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:13.310332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3488b-dd48-4e6e-8e83-0ba2b68f5414 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:13.315672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a87964-1584-4449-af1d-417903c82513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:13.316355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a87964-1584-4449-af1d-417903c82513 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:13.316636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a87964-1584-4449-af1d-417903c82513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:13.316906 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a87964-1584-4449-af1d-417903c82513 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:13.317049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4c149-85cf-4471-850c-ae4809dc6000 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:13.317718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4c149-85cf-4471-850c-ae4809dc6000 service swift] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:13.317822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a87964-1584-4449-af1d-417903c82513 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:16:13.318003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4c149-85cf-4471-850c-ae4809dc6000 service swift] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:13.318133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a87964-1584-4449-af1d-417903c82513 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:13.318320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac4c149-85cf-4471-850c-ae4809dc6000 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:13.320162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a87964-1584-4449-af1d-417903c82513 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:13.336977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1254/2494] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:16:13.337343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a87964-1584-4449-af1d-417903c82513 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:13.343773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:13.348355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a87964-1584-4449-af1d-417903c82513 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:13.355895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e44a5-5d17-4bce-a4a3-268670b17369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:13.355895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e44a5-5d17-4bce-a4a3-268670b17369 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:13.355895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e44a5-5d17-4bce-a4a3-268670b17369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:13.355895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e44a5-5d17-4bce-a4a3-268670b17369 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:13.356719 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e44a5-5d17-4bce-a4a3-268670b17369 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:13.359394 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e44a5-5d17-4bce-a4a3-268670b17369 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:13.381954 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac4c149-85cf-4471-850c-ae4809dc6000 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:13.383941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac4c149-85cf-4471-850c-ae4809dc6000 service swift] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:13.391504 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1255/2495] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:16:13 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:16:13.392585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1241/2496] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:16:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 19:16:13.395539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e44a5-5d17-4bce-a4a3-268670b17369 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:13.507562 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e44a5-5d17-4bce-a4a3-268670b17369 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:13.512638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b358b-2de0-41c3-af27-1c26f421e96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:13.513423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b358b-2de0-41c3-af27-1c26f421e96e admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:13.513646 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b358b-2de0-41c3-af27-1c26f421e96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:13.513893 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b358b-2de0-41c3-af27-1c26f421e96e admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:13.514512 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b358b-2de0-41c3-af27-1c26f421e96e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:13.516423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b358b-2de0-41c3-af27-1c26f421e96e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:13.542289 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1256/2497] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:16:13 2022] POST /identity/v3/users => generated 292 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:16:13.554503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4c149-85cf-4471-850c-ae4809dc6000 service swift] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:13.566264 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac4c149-85cf-4471-850c-ae4809dc6000 service swift] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:13.572630 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c3ed6-de88-46d7-9e65-8893338bf046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:13.573381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c3ed6-de88-46d7-9e65-8893338bf046 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:13.573757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c3ed6-de88-46d7-9e65-8893338bf046 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:13.574118 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9c3ed6-de88-46d7-9e65-8893338bf046 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:13.574829 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c3ed6-de88-46d7-9e65-8893338bf046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:13.576518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9c3ed6-de88-46d7-9e65-8893338bf046 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:13.584514 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:13.589404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1242/2498] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:16:13.593171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b358b-2de0-41c3-af27-1c26f421e96e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:13.603958 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b358b-2de0-41c3-af27-1c26f421e96e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:13.610293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bdb3e7-77e6-4c7b-ad31-9dc9eedd9c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:13.610913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdb3e7-77e6-4c7b-ad31-9dc9eedd9c3e admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:13.611192 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdb3e7-77e6-4c7b-ad31-9dc9eedd9c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:13.611493 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bdb3e7-77e6-4c7b-ad31-9dc9eedd9c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:13.612159 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bdb3e7-77e6-4c7b-ad31-9dc9eedd9c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:13.613855 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bdb3e7-77e6-4c7b-ad31-9dc9eedd9c3e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:13.626022 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1257/2499] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:16:13.630440 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c3ed6-de88-46d7-9e65-8893338bf046 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:13.642741 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9c3ed6-de88-46d7-9e65-8893338bf046 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:13.647603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5925e-159b-4de2-9797-2f6b374f4fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:13.648260 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5925e-159b-4de2-9797-2f6b374f4fe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:13.648538 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5925e-159b-4de2-9797-2f6b374f4fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:13.648817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd5925e-159b-4de2-9797-2f6b374f4fe9 admin admin] PATH_INFO: `/v3/projects/fe8ee0bdd09b45599a7c19798660aac0/users/a07fb9a216ac4683852fc6cae7c10b99/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:13.671266 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5925e-159b-4de2-9797-2f6b374f4fe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe8ee0bdd09b45599a7c19798660aac0, user_id=a07fb9a216ac4683852fc6cae7c10b99, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:13.675917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd5925e-159b-4de2-9797-2f6b374f4fe9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:13.699433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1243/2500] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:16:13 2022] PUT /identity/v3/projects/fe8ee0bdd09b45599a7c19798660aac0/users/a07fb9a216ac4683852fc6cae7c10b99/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:16:13.704980 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18419c30-2ca8-42a0-86d2-7e9eeed69fa7 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:13.705506 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18419c30-2ca8-42a0-86d2-7e9eeed69fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:13.705857 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18419c30-2ca8-42a0-86d2-7e9eeed69fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:13.728137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-18419c30-2ca8-42a0-86d2-7e9eeed69fa7 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:13.729999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-18419c30-2ca8-42a0-86d2-7e9eeed69fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:13.743205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5925e-159b-4de2-9797-2f6b374f4fe9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:13.744982 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7dd5925e-159b-4de2-9797-2f6b374f4fe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:13.836598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd5925e-159b-4de2-9797-2f6b374f4fe9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:13.846963 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefa09c-9782-49ce-8465-6475e2532e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:13.848310 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefa09c-9782-49ce-8465-6475e2532e2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:13.848654 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefa09c-9782-49ce-8465-6475e2532e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:13.848990 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefa09c-9782-49ce-8465-6475e2532e2e admin admin] PATH_INFO: `/v3/users/0c4a21ea6a2c4870a2fe009027c12f9d` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:13.850383 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefa09c-9782-49ce-8465-6475e2532e2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c4a21ea6a2c4870a2fe009027c12f9d)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:13.852751 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefa09c-9782-49ce-8465-6475e2532e2e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:13.876269 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1258/2501] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:16:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:13.881900 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18419c30-2ca8-42a0-86d2-7e9eeed69fa7 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:13.884464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-18419c30-2ca8-42a0-86d2-7e9eeed69fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:13.942132 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eeefa09c-9782-49ce-8465-6475e2532e2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c4a21ea6a2c4870a2fe009027c12f9d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:13.958744 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eeefa09c-9782-49ce-8465-6475e2532e2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c4a21ea6a2c4870a2fe009027c12f9d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:13.985609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eeefa09c-9782-49ce-8465-6475e2532e2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c4a21ea6a2c4870a2fe009027c12f9d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:13.986426 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eeefa09c-9782-49ce-8465-6475e2532e2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c4a21ea6a2c4870a2fe009027c12f9d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:13.989000 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eeefa09c-9782-49ce-8465-6475e2532e2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c4a21ea6a2c4870a2fe009027c12f9d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:14.018892 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1244/2502] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:16:13 2022] DELETE /identity/v3/users/0c4a21ea6a2c4870a2fe009027c12f9d => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:16:14.020219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7f219-92d2-4a80-8c7b-8d33d6b333b0 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:14.020523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7f219-92d2-4a80-8c7b-8d33d6b333b0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:14.020889 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e7f219-92d2-4a80-8c7b-8d33d6b333b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:14.035244 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18419c30-2ca8-42a0-86d2-7e9eeed69fa7 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:14.047219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a3e7f219-92d2-4a80-8c7b-8d33d6b333b0 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:14.048436 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a3e7f219-92d2-4a80-8c7b-8d33d6b333b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:14.051692 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:14.109219 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcf061-11a4-45f4-af2f-70472d11f3f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:14.110197 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcf061-11a4-45f4-af2f-70472d11f3f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:14.110710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcf061-11a4-45f4-af2f-70472d11f3f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:14.111165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dcf061-11a4-45f4-af2f-70472d11f3f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:14.114775 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-47dcf061-11a4-45f4-af2f-70472d11f3f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:14.138290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:14.169475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1245/2503] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:14.184689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7f219-92d2-4a80-8c7b-8d33d6b333b0 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:14.237372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a3e7f219-92d2-4a80-8c7b-8d33d6b333b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:14.266930 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e7f219-92d2-4a80-8c7b-8d33d6b333b0 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:14.279324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ad6a7-6bd6-4b19-a547-7617736924a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:14.280467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ad6a7-6bd6-4b19-a547-7617736924a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:14.280919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ad6a7-6bd6-4b19-a547-7617736924a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:14.281512 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ad6a7-6bd6-4b19-a547-7617736924a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:14.301093 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dcf061-11a4-45f4-af2f-70472d11f3f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:14.304021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dcf061-11a4-45f4-af2f-70472d11f3f9 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:14.313383 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ad6a7-6bd6-4b19-a547-7617736924a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:14.332454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1259/2504] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:16:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:16:14.332791 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ad6a7-6bd6-4b19-a547-7617736924a5 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:14.354880 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1246/2505] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:16:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:16:14.770704 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcf061-11a4-45f4-af2f-70472d11f3f9 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:14.772197 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-47dcf061-11a4-45f4-af2f-70472d11f3f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:14.795539 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dcf061-11a4-45f4-af2f-70472d11f3f9 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:14.802872 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a1553-4d9f-4056-bf8f-a0f829264a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:14.803605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a1553-4d9f-4056-bf8f-a0f829264a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:14.803967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a1553-4d9f-4056-bf8f-a0f829264a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:14.804207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a1553-4d9f-4056-bf8f-a0f829264a7a admin admin] PATH_INFO: `/v3/projects/7450d4b1a8fe4762b7f8cbfaf7163bce` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:14.812937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a1553-4d9f-4056-bf8f-a0f829264a7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7450d4b1a8fe4762b7f8cbfaf7163bce)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:14.815233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a1553-4d9f-4056-bf8f-a0f829264a7a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:14.948187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9e0a1553-4d9f-4056-bf8f-a0f829264a7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7450d4b1a8fe4762b7f8cbfaf7163bce', 'request_id': 'req-9e0a1553-4d9f-4056-bf8f-a0f829264a7a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:14.959929 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9e0a1553-4d9f-4056-bf8f-a0f829264a7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7450d4b1a8fe4762b7f8cbfaf7163bce', 'request_id': 'req-9e0a1553-4d9f-4056-bf8f-a0f829264a7a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:14.978197 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9e0a1553-4d9f-4056-bf8f-a0f829264a7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7450d4b1a8fe4762b7f8cbfaf7163bce', 'request_id': 'req-9e0a1553-4d9f-4056-bf8f-a0f829264a7a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:15.016500 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9e0a1553-4d9f-4056-bf8f-a0f829264a7a admin admin] The token cache is being invalidate because project 7450d4b1a8fe4762b7f8cbfaf7163bce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:16:15.018237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9e0a1553-4d9f-4056-bf8f-a0f829264a7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:15.020877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9e0a1553-4d9f-4056-bf8f-a0f829264a7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:15.030545 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1260/2506] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:16:14 2022] DELETE /identity/v3/projects/7450d4b1a8fe4762b7f8cbfaf7163bce => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:16:16.306099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ad6a7-6bd6-4b19-a547-7617736924a5 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:16.306099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-466ad6a7-6bd6-4b19-a547-7617736924a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:16.329483 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ad6a7-6bd6-4b19-a547-7617736924a5 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:16.411057 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f6ace-f9d7-43e2-bf1f-d2765a1acccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:16.411913 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f6ace-f9d7-43e2-bf1f-d2765a1acccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:16.412329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f6ace-f9d7-43e2-bf1f-d2765a1acccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:16.412796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f6ace-f9d7-43e2-bf1f-d2765a1acccd admin admin] PATH_INFO: `/v3/users/93aabb087e424130bf7ece089a575e2d` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:16.414241 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f6ace-f9d7-43e2-bf1f-d2765a1acccd admin admin] RBAC: Authorizing `identity:delete_user(user_id=93aabb087e424130bf7ece089a575e2d)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:16.416547 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f6ace-f9d7-43e2-bf1f-d2765a1acccd admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:16.546107 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-498f6ace-f9d7-43e2-bf1f-d2765a1acccd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93aabb087e424130bf7ece089a575e2d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:16.559662 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-498f6ace-f9d7-43e2-bf1f-d2765a1acccd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93aabb087e424130bf7ece089a575e2d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:16.593331 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-498f6ace-f9d7-43e2-bf1f-d2765a1acccd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93aabb087e424130bf7ece089a575e2d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:16.595743 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-498f6ace-f9d7-43e2-bf1f-d2765a1acccd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93aabb087e424130bf7ece089a575e2d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:16.597119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-498f6ace-f9d7-43e2-bf1f-d2765a1acccd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93aabb087e424130bf7ece089a575e2d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:16.620892 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1247/2507] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:16:16 2022] DELETE /identity/v3/users/93aabb087e424130bf7ece089a575e2d => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:16:16.625688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de19ad8-fb6b-4956-96b7-83ea434d44f7 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:16.626812 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de19ad8-fb6b-4956-96b7-83ea434d44f7 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:16.627337 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de19ad8-fb6b-4956-96b7-83ea434d44f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:16.667533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2de19ad8-fb6b-4956-96b7-83ea434d44f7 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:16.668369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2de19ad8-fb6b-4956-96b7-83ea434d44f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:16.781629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:16.850758 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1261/2508] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:16:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:16.945007 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f6ace-f9d7-43e2-bf1f-d2765a1acccd admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:16.946530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-498f6ace-f9d7-43e2-bf1f-d2765a1acccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:17.055493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f6ace-f9d7-43e2-bf1f-d2765a1acccd admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:17.064520 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b0f360-d9a3-408c-ba5e-5056ede3f9ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:17.065595 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0f360-d9a3-408c-ba5e-5056ede3f9ea service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:17.066221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0f360-d9a3-408c-ba5e-5056ede3f9ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:17.067927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b0f360-d9a3-408c-ba5e-5056ede3f9ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:17.126851 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0f360-d9a3-408c-ba5e-5056ede3f9ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:17.128460 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b0f360-d9a3-408c-ba5e-5056ede3f9ea service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:17.154076 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1248/2509] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:16:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:16:18.083482 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de19ad8-fb6b-4956-96b7-83ea434d44f7 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:18.085457 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2de19ad8-fb6b-4956-96b7-83ea434d44f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:18.116696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de19ad8-fb6b-4956-96b7-83ea434d44f7 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:18.127144 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8136be3-5b18-4bc0-9668-cd2c534cc39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:18.127946 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8136be3-5b18-4bc0-9668-cd2c534cc39b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:18.128685 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8136be3-5b18-4bc0-9668-cd2c534cc39b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:18.129241 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8136be3-5b18-4bc0-9668-cd2c534cc39b admin admin] PATH_INFO: `/v3/projects/e6e7499326f34996a62da00eb5e21705` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:18.130942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8136be3-5b18-4bc0-9668-cd2c534cc39b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6e7499326f34996a62da00eb5e21705)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:18.133234 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8136be3-5b18-4bc0-9668-cd2c534cc39b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:18.225224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b8136be3-5b18-4bc0-9668-cd2c534cc39b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6e7499326f34996a62da00eb5e21705', 'request_id': 'req-b8136be3-5b18-4bc0-9668-cd2c534cc39b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:18.238298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b8136be3-5b18-4bc0-9668-cd2c534cc39b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6e7499326f34996a62da00eb5e21705', 'request_id': 'req-b8136be3-5b18-4bc0-9668-cd2c534cc39b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:18.247559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b8136be3-5b18-4bc0-9668-cd2c534cc39b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6e7499326f34996a62da00eb5e21705', 'request_id': 'req-b8136be3-5b18-4bc0-9668-cd2c534cc39b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:18.293288 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b8136be3-5b18-4bc0-9668-cd2c534cc39b admin admin] The token cache is being invalidate because project e6e7499326f34996a62da00eb5e21705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:16:18.293917 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b8136be3-5b18-4bc0-9668-cd2c534cc39b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:18.296633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b8136be3-5b18-4bc0-9668-cd2c534cc39b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:18.310135 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1262/2510] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:16:18 2022] DELETE /identity/v3/projects/e6e7499326f34996a62da00eb5e21705 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:16:22.301064 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a4187-b024-45fb-8c8c-cfa6de6e4492 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:22.301699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a4187-b024-45fb-8c8c-cfa6de6e4492 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:22.301699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a4187-b024-45fb-8c8c-cfa6de6e4492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:22.329288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:22.369824 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-d34a4187-b024-45fb-8c8c-cfa6de6e4492 None None] MFA Rules not processed for user `a07fb9a216ac4683852fc6cae7c10b99`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:22.372052 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d34a4187-b024-45fb-8c8c-cfa6de6e4492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:22.560200 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1249/2511] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:16:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:22.560858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8136be3-5b18-4bc0-9668-cd2c534cc39b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:22.562364 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b8136be3-5b18-4bc0-9668-cd2c534cc39b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:22.694458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8136be3-5b18-4bc0-9668-cd2c534cc39b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:22.701757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90384efb-e947-4a34-86e9-dde40ed9f315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:22.702821 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90384efb-e947-4a34-86e9-dde40ed9f315 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:22.703255 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90384efb-e947-4a34-86e9-dde40ed9f315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:22.703686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90384efb-e947-4a34-86e9-dde40ed9f315 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:22.704478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90384efb-e947-4a34-86e9-dde40ed9f315 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:22.706914 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90384efb-e947-4a34-86e9-dde40ed9f315 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:22.759686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1263/2512] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:16:22 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:16:22.764322 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a4187-b024-45fb-8c8c-cfa6de6e4492 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:22.784003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a4187-b024-45fb-8c8c-cfa6de6e4492 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:22.932261 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce383ebe-f0ad-4a76-bba9-852bb4f84dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:22.932878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce383ebe-f0ad-4a76-bba9-852bb4f84dfa admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:22.933156 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce383ebe-f0ad-4a76-bba9-852bb4f84dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:22.933474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce383ebe-f0ad-4a76-bba9-852bb4f84dfa admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:22.934108 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce383ebe-f0ad-4a76-bba9-852bb4f84dfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:22.940276 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce383ebe-f0ad-4a76-bba9-852bb4f84dfa admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:22.985590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1250/2513] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:16:22 2022] POST /identity/v3/users => generated 298 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:16:22.987537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90384efb-e947-4a34-86e9-dde40ed9f315 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:23.001177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90384efb-e947-4a34-86e9-dde40ed9f315 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:23.023670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e27a96-e516-494b-bac0-cb4042d1fd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:23.024577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e27a96-e516-494b-bac0-cb4042d1fd66 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:23.025078 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e27a96-e516-494b-bac0-cb4042d1fd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:23.025599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e27a96-e516-494b-bac0-cb4042d1fd66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:23.026423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e27a96-e516-494b-bac0-cb4042d1fd66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:23.028215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e27a96-e516-494b-bac0-cb4042d1fd66 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:23.050390 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1264/2514] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:16:23.052173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce383ebe-f0ad-4a76-bba9-852bb4f84dfa admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:23.066523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce383ebe-f0ad-4a76-bba9-852bb4f84dfa admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:23.072506 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babd941-d668-4725-85fc-24f708f57e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:23.073380 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babd941-d668-4725-85fc-24f708f57e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:23.073701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babd941-d668-4725-85fc-24f708f57e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:23.073981 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7babd941-d668-4725-85fc-24f708f57e09 admin admin] PATH_INFO: `/v3/projects/1c877d864d984a3eb0a752a6d9c4e697/users/e3c28a6f05274f10af0944d6d209f098/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:23.113910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7babd941-d668-4725-85fc-24f708f57e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c877d864d984a3eb0a752a6d9c4e697, user_id=e3c28a6f05274f10af0944d6d209f098, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:23.117317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7babd941-d668-4725-85fc-24f708f57e09 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:23.158082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1251/2515] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:16:23 2022] PUT /identity/v3/projects/1c877d864d984a3eb0a752a6d9c4e697/users/e3c28a6f05274f10af0944d6d209f098/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:16:23.159185 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e27a96-e516-494b-bac0-cb4042d1fd66 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:23.180683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e27a96-e516-494b-bac0-cb4042d1fd66 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:23.394761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:23.441230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1850573-3ad1-4214-acf0-725724b3fa96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:23.441843 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1850573-3ad1-4214-acf0-725724b3fa96 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:23.442114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1850573-3ad1-4214-acf0-725724b3fa96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:23.442385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1850573-3ad1-4214-acf0-725724b3fa96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:23.443026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1850573-3ad1-4214-acf0-725724b3fa96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:23.444677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1850573-3ad1-4214-acf0-725724b3fa96 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:23.470095 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1265/2516] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:16:23.470773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babd941-d668-4725-85fc-24f708f57e09 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:23.485521 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7babd941-d668-4725-85fc-24f708f57e09 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:23.503929 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6478316c-10e5-4a75-b5d1-c77af16cdd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:23.505351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478316c-10e5-4a75-b5d1-c77af16cdd4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:23.505883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478316c-10e5-4a75-b5d1-c77af16cdd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:23.506467 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6478316c-10e5-4a75-b5d1-c77af16cdd4f admin admin] PATH_INFO: `/v3/projects/1c877d864d984a3eb0a752a6d9c4e697/users/e3c28a6f05274f10af0944d6d209f098/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:23.509231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478316c-10e5-4a75-b5d1-c77af16cdd4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c877d864d984a3eb0a752a6d9c4e697, user_id=e3c28a6f05274f10af0944d6d209f098, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:23.512224 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6478316c-10e5-4a75-b5d1-c77af16cdd4f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:23.525321 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1252/2517] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:16:23 2022] PUT /identity/v3/projects/1c877d864d984a3eb0a752a6d9c4e697/users/e3c28a6f05274f10af0944d6d209f098/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:16:26.511870 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1850573-3ad1-4214-acf0-725724b3fa96 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:26.515114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f1850573-3ad1-4214-acf0-725724b3fa96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:26.693505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1850573-3ad1-4214-acf0-725724b3fa96 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:26.804319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17a55d-e192-4610-a7db-e30e632571f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:26.804964 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17a55d-e192-4610-a7db-e30e632571f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:26.805250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17a55d-e192-4610-a7db-e30e632571f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:26.805526 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17a55d-e192-4610-a7db-e30e632571f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:26.809254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7d17a55d-e192-4610-a7db-e30e632571f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:26.975278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17a55d-e192-4610-a7db-e30e632571f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:26.979783 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17a55d-e192-4610-a7db-e30e632571f0 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:26.997282 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1266/2518] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:16:26 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:16:32.667200 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d2e76-32a1-415d-b1df-0bdb8f776886 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:32.672505 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d2e76-32a1-415d-b1df-0bdb8f776886 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:32.673043 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d2e76-32a1-415d-b1df-0bdb8f776886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:32.749551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-767d2e76-32a1-415d-b1df-0bdb8f776886 None None] MFA Rules not processed for user `e3c28a6f05274f10af0944d6d209f098`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:32.750749 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-767d2e76-32a1-415d-b1df-0bdb8f776886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:32.875796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:32.992320 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1253/2519] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:16:32 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:36.992087 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=77 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:40.082081 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17a55d-e192-4610-a7db-e30e632571f0 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:40.083561 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7d17a55d-e192-4610-a7db-e30e632571f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:40.148177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17a55d-e192-4610-a7db-e30e632571f0 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:40.152517 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f2768-a623-4403-93a2-bd9e6deea512 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1863336fecad4987acdfd0b3f7b2f70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:40.153230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f2768-a623-4403-93a2-bd9e6deea512 service glance] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:40.154190 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f2768-a623-4403-93a2-bd9e6deea512 service glance] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:40.154542 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f2768-a623-4403-93a2-bd9e6deea512 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:40.231341 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f2768-a623-4403-93a2-bd9e6deea512 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:40.233753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f2768-a623-4403-93a2-bd9e6deea512 service glance] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:40.246611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1267/2520] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:16:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:16:40.395416 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424e3d9-9ff1-4fc2-8ef3-07d536feecb9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:40.395904 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424e3d9-9ff1-4fc2-8ef3-07d536feecb9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:40.396040 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5424e3d9-9ff1-4fc2-8ef3-07d536feecb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:40.416519 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-5424e3d9-9ff1-4fc2-8ef3-07d536feecb9 None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:40.417743 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5424e3d9-9ff1-4fc2-8ef3-07d536feecb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:40.479785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1254/2521] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 19:16:40 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:40.492293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f2768-a623-4403-93a2-bd9e6deea512 service glance] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:40.493727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-714f2768-a623-4403-93a2-bd9e6deea512 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:40.564536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f2768-a623-4403-93a2-bd9e6deea512 service glance] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:40.569995 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526a59c-6441-472a-9837-3e2d926aafdf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:40.570601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526a59c-6441-472a-9837-3e2d926aafdf service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:40.570876 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526a59c-6441-472a-9837-3e2d926aafdf service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:40.571143 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526a59c-6441-472a-9837-3e2d926aafdf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:40.582292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526a59c-6441-472a-9837-3e2d926aafdf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:40.583936 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526a59c-6441-472a-9837-3e2d926aafdf service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:40.593078 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1268/2522] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:16:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:16:42.276593 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437c004-74f0-420f-9068-484f0da1be02 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:42.277097 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437c004-74f0-420f-9068-484f0da1be02 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:42.277244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6437c004-74f0-420f-9068-484f0da1be02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:42.298061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-6437c004-74f0-420f-9068-484f0da1be02 None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:42.299527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6437c004-74f0-420f-9068-484f0da1be02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:42.307710 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1255/2523] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 31 19:16:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:42.323211 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526a59c-6441-472a-9837-3e2d926aafdf service swift] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:42.337773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526a59c-6441-472a-9837-3e2d926aafdf service swift] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:42.343742 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205588d2-a323-48ff-8703-6d35eb1d1c64 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:42.344666 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205588d2-a323-48ff-8703-6d35eb1d1c64 service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:42.345050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205588d2-a323-48ff-8703-6d35eb1d1c64 service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:42.345690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205588d2-a323-48ff-8703-6d35eb1d1c64 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:42.359728 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205588d2-a323-48ff-8703-6d35eb1d1c64 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:42.361470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205588d2-a323-48ff-8703-6d35eb1d1c64 service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:42.373931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1269/2524] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:16:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:16:44.751033 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437c004-74f0-420f-9068-484f0da1be02 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:44.752975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6437c004-74f0-420f-9068-484f0da1be02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:44.833564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6437c004-74f0-420f-9068-484f0da1be02 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:44.839133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1cfa6d-64cc-4978-af2a-24788c145feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:44.839780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cfa6d-64cc-4978-af2a-24788c145feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:44.840058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cfa6d-64cc-4978-af2a-24788c145feb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:44.840336 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1cfa6d-64cc-4978-af2a-24788c145feb admin admin] PATH_INFO: `/v3/users/e33c120ef02f4f6e8bea53107d90034b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:44.841597 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cfa6d-64cc-4978-af2a-24788c145feb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e33c120ef02f4f6e8bea53107d90034b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:44.843529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1cfa6d-64cc-4978-af2a-24788c145feb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:44.907025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-cf1cfa6d-64cc-4978-af2a-24788c145feb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e33c120ef02f4f6e8bea53107d90034b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:44.914884 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-cf1cfa6d-64cc-4978-af2a-24788c145feb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e33c120ef02f4f6e8bea53107d90034b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:44.930101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:44.931568 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-cf1cfa6d-64cc-4978-af2a-24788c145feb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e33c120ef02f4f6e8bea53107d90034b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:44.932726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-cf1cfa6d-64cc-4978-af2a-24788c145feb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e33c120ef02f4f6e8bea53107d90034b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:44.933520 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-cf1cfa6d-64cc-4978-af2a-24788c145feb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e33c120ef02f4f6e8bea53107d90034b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:44.952686 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1256/2525] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:16:44 2022] DELETE /identity/v3/users/e33c120ef02f4f6e8bea53107d90034b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:16:45.148882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205588d2-a323-48ff-8703-6d35eb1d1c64 service swift] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:45.174896 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205588d2-a323-48ff-8703-6d35eb1d1c64 service swift] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:45.215329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:45.244030 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22180e2-18a5-4053-8563-0a58bfbc4273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:45.244773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22180e2-18a5-4053-8563-0a58bfbc4273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:45.245118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22180e2-18a5-4053-8563-0a58bfbc4273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:45.245487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22180e2-18a5-4053-8563-0a58bfbc4273 admin admin] PATH_INFO: `/v3/projects/426e58f469324233b3514e866529eafa` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:45.246679 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22180e2-18a5-4053-8563-0a58bfbc4273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=426e58f469324233b3514e866529eafa)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:45.249235 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22180e2-18a5-4053-8563-0a58bfbc4273 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:45.312167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a22180e2-18a5-4053-8563-0a58bfbc4273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '426e58f469324233b3514e866529eafa', 'request_id': 'req-a22180e2-18a5-4053-8563-0a58bfbc4273'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:45.319190 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a22180e2-18a5-4053-8563-0a58bfbc4273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '426e58f469324233b3514e866529eafa', 'request_id': 'req-a22180e2-18a5-4053-8563-0a58bfbc4273'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:45.325088 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a22180e2-18a5-4053-8563-0a58bfbc4273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '426e58f469324233b3514e866529eafa', 'request_id': 'req-a22180e2-18a5-4053-8563-0a58bfbc4273'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:45.342112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a22180e2-18a5-4053-8563-0a58bfbc4273 admin admin] The token cache is being invalidate because project 426e58f469324233b3514e866529eafa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:16:45.342560 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a22180e2-18a5-4053-8563-0a58bfbc4273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:45.343272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a22180e2-18a5-4053-8563-0a58bfbc4273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:45.345022 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1270/2526] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:16:45 2022] DELETE /identity/v3/projects/426e58f469324233b3514e866529eafa => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:16:45.349923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d23c87-2463-4728-874f-5b20cbc8aeea None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:45.350212 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d23c87-2463-4728-874f-5b20cbc8aeea None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:45.350488 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d23c87-2463-4728-874f-5b20cbc8aeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:45.376787 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c3d23c87-2463-4728-874f-5b20cbc8aeea None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:45.376787 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c3d23c87-2463-4728-874f-5b20cbc8aeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:45.463880 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1257/2527] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:45.522671 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13360cd8-6d47-4026-8d37-1fef9c0f281b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:45.523028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13360cd8-6d47-4026-8d37-1fef9c0f281b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:45.523229 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13360cd8-6d47-4026-8d37-1fef9c0f281b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:45.547917 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-13360cd8-6d47-4026-8d37-1fef9c0f281b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:45.549546 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-13360cd8-6d47-4026-8d37-1fef9c0f281b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:45.558524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1271/2528] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:16:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:45.562433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d23c87-2463-4728-874f-5b20cbc8aeea None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:45.572935 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d23c87-2463-4728-874f-5b20cbc8aeea None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:45.578294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97500aa5-34a3-4a85-a0a0-0c623fac38f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:45.578900 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97500aa5-34a3-4a85-a0a0-0c623fac38f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:45.579182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97500aa5-34a3-4a85-a0a0-0c623fac38f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:45.579450 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97500aa5-34a3-4a85-a0a0-0c623fac38f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:45.580088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97500aa5-34a3-4a85-a0a0-0c623fac38f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:16:45.580390 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97500aa5-34a3-4a85-a0a0-0c623fac38f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:45.582135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97500aa5-34a3-4a85-a0a0-0c623fac38f1 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:45.595421 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1258/2529] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:16:45.599634 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13360cd8-6d47-4026-8d37-1fef9c0f281b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:45.611339 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13360cd8-6d47-4026-8d37-1fef9c0f281b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:45.616081 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a6687-69ca-4558-b07e-d8182ccc2b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:45.616679 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a6687-69ca-4558-b07e-d8182ccc2b39 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:45.616953 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a6687-69ca-4558-b07e-d8182ccc2b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:45.617240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1a6687-69ca-4558-b07e-d8182ccc2b39 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:45.618017 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a6687-69ca-4558-b07e-d8182ccc2b39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:45.619898 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1a6687-69ca-4558-b07e-d8182ccc2b39 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:45.653230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1272/2530] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:16:45 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:16:45.656006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97500aa5-34a3-4a85-a0a0-0c623fac38f1 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:45.671468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97500aa5-34a3-4a85-a0a0-0c623fac38f1 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:45.739403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90076302-bcfe-4f91-96ad-b723adb7ee7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:45.739944 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90076302-bcfe-4f91-96ad-b723adb7ee7d admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:45.740215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90076302-bcfe-4f91-96ad-b723adb7ee7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:45.740538 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90076302-bcfe-4f91-96ad-b723adb7ee7d admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:45.741204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90076302-bcfe-4f91-96ad-b723adb7ee7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:45.743523 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90076302-bcfe-4f91-96ad-b723adb7ee7d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:45.768539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1259/2531] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:16:45 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:16:45.772192 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a6687-69ca-4558-b07e-d8182ccc2b39 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:45.784082 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1a6687-69ca-4558-b07e-d8182ccc2b39 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:45.789376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef37a8-97be-4a9f-8192-508856f13323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:45.790092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef37a8-97be-4a9f-8192-508856f13323 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:45.790481 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef37a8-97be-4a9f-8192-508856f13323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:45.790862 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ef37a8-97be-4a9f-8192-508856f13323 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:45.791575 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef37a8-97be-4a9f-8192-508856f13323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:45.793379 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ef37a8-97be-4a9f-8192-508856f13323 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:45.805938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1273/2532] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:16:45.817839 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90076302-bcfe-4f91-96ad-b723adb7ee7d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:45.831288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90076302-bcfe-4f91-96ad-b723adb7ee7d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:45.837892 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e208ed7a-a678-4b10-9d78-ff03a1062180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:45.838428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208ed7a-a678-4b10-9d78-ff03a1062180 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:45.838547 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208ed7a-a678-4b10-9d78-ff03a1062180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:45.838806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e208ed7a-a678-4b10-9d78-ff03a1062180 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:45.839447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e208ed7a-a678-4b10-9d78-ff03a1062180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:45.841641 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e208ed7a-a678-4b10-9d78-ff03a1062180 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:45.853313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1260/2533] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:16:45.858446 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef37a8-97be-4a9f-8192-508856f13323 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:45.868617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ef37a8-97be-4a9f-8192-508856f13323 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:45.873455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cd3f7-63ab-4fbe-9430-60575b55220b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:45.874135 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cd3f7-63ab-4fbe-9430-60575b55220b admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:45.874398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cd3f7-63ab-4fbe-9430-60575b55220b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:45.874667 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80cd3f7-63ab-4fbe-9430-60575b55220b admin admin] PATH_INFO: `/v3/projects/d7391967dbca463b9b0a3ac3753cf96b/users/10113de40dec47ccb721640f2faafe71/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:46.018104 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80cd3f7-63ab-4fbe-9430-60575b55220b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7391967dbca463b9b0a3ac3753cf96b, user_id=10113de40dec47ccb721640f2faafe71, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:46.020730 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80cd3f7-63ab-4fbe-9430-60575b55220b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:46.036422 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1274/2534] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:16:45 2022] PUT /identity/v3/projects/d7391967dbca463b9b0a3ac3753cf96b/users/10113de40dec47ccb721640f2faafe71/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:16:46.046801 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aab52c-86ee-49de-9740-77e456103358 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:46.047354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aab52c-86ee-49de-9740-77e456103358 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:46.047732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aab52c-86ee-49de-9740-77e456103358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:46.078913 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-42aab52c-86ee-49de-9740-77e456103358 None None] MFA Rules not processed for user `10113de40dec47ccb721640f2faafe71`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:46.080238 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-42aab52c-86ee-49de-9740-77e456103358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:46.135949 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:46.165448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1261/2535] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:16:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:46.178286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cd3f7-63ab-4fbe-9430-60575b55220b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:46.180329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d80cd3f7-63ab-4fbe-9430-60575b55220b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:46.252007 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80cd3f7-63ab-4fbe-9430-60575b55220b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:46.255820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546f682-d554-4c50-ac40-284613e4cb30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:46.256478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546f682-d554-4c50-ac40-284613e4cb30 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:46.256761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546f682-d554-4c50-ac40-284613e4cb30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:46.257028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f546f682-d554-4c50-ac40-284613e4cb30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:46.270432 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546f682-d554-4c50-ac40-284613e4cb30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:46.272022 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f546f682-d554-4c50-ac40-284613e4cb30 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:46.276986 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:46.284727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1275/2536] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:16:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:16:49.041530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca547804-6669-4b1e-949e-84ef86c24c2f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:49.042058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca547804-6669-4b1e-949e-84ef86c24c2f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:49.042058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca547804-6669-4b1e-949e-84ef86c24c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:49.061110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ca547804-6669-4b1e-949e-84ef86c24c2f None None] MFA Rules not processed for user `a07fb9a216ac4683852fc6cae7c10b99`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:49.062336 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ca547804-6669-4b1e-949e-84ef86c24c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:49.118947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1262/2537] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:16:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:50.019815 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546f682-d554-4c50-ac40-284613e4cb30 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:50.090540 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f546f682-d554-4c50-ac40-284613e4cb30 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:50.095033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb350cc5-f8bf-4be8-84af-215d8ae73abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:50.095652 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb350cc5-f8bf-4be8-84af-215d8ae73abe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:50.095931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb350cc5-f8bf-4be8-84af-215d8ae73abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:50.096201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb350cc5-f8bf-4be8-84af-215d8ae73abe admin admin] PATH_INFO: `/v3/users/10113de40dec47ccb721640f2faafe71` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:50.097348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb350cc5-f8bf-4be8-84af-215d8ae73abe admin admin] RBAC: Authorizing `identity:delete_user(user_id=10113de40dec47ccb721640f2faafe71)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:50.099294 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb350cc5-f8bf-4be8-84af-215d8ae73abe admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:50.148404 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-cb350cc5-f8bf-4be8-84af-215d8ae73abe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10113de40dec47ccb721640f2faafe71'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:50.155414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-cb350cc5-f8bf-4be8-84af-215d8ae73abe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10113de40dec47ccb721640f2faafe71'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:50.163065 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-cb350cc5-f8bf-4be8-84af-215d8ae73abe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10113de40dec47ccb721640f2faafe71'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:50.163735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-cb350cc5-f8bf-4be8-84af-215d8ae73abe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10113de40dec47ccb721640f2faafe71'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:50.176978 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-cb350cc5-f8bf-4be8-84af-215d8ae73abe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10113de40dec47ccb721640f2faafe71'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:50.183573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1276/2538] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:16:50 2022] DELETE /identity/v3/users/10113de40dec47ccb721640f2faafe71 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:16:50.188720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a971a-5df6-4ffb-9093-f28c8ab0cf03 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:50.189232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a971a-5df6-4ffb-9093-f28c8ab0cf03 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:50.189348 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093a971a-5df6-4ffb-9093-f28c8ab0cf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:50.209395 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-093a971a-5df6-4ffb-9093-f28c8ab0cf03 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:50.210648 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-093a971a-5df6-4ffb-9093-f28c8ab0cf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:50.271306 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1263/2539] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:16:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:16:50.282097 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb350cc5-f8bf-4be8-84af-215d8ae73abe admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:50.282847 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cb350cc5-f8bf-4be8-84af-215d8ae73abe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:50.341045 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb350cc5-f8bf-4be8-84af-215d8ae73abe admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:50.344590 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88be942-e12b-4b96-b3a0-ad8a046694ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:50.345206 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88be942-e12b-4b96-b3a0-ad8a046694ac service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:50.345500 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88be942-e12b-4b96-b3a0-ad8a046694ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:50.345774 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88be942-e12b-4b96-b3a0-ad8a046694ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:50.355727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88be942-e12b-4b96-b3a0-ad8a046694ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:50.357698 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88be942-e12b-4b96-b3a0-ad8a046694ac service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:50.368139 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1277/2540] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:16:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:16:50.655522 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a971a-5df6-4ffb-9093-f28c8ab0cf03 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:50.657530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-093a971a-5df6-4ffb-9093-f28c8ab0cf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:50.673015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093a971a-5df6-4ffb-9093-f28c8ab0cf03 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:50.678727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819d233-4251-4984-812c-3afe96372097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:50.679547 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819d233-4251-4984-812c-3afe96372097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:50.679936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819d233-4251-4984-812c-3afe96372097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:50.680326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9819d233-4251-4984-812c-3afe96372097 admin admin] PATH_INFO: `/v3/projects/d7391967dbca463b9b0a3ac3753cf96b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:50.681772 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819d233-4251-4984-812c-3afe96372097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7391967dbca463b9b0a3ac3753cf96b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:50.683956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9819d233-4251-4984-812c-3afe96372097 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:50.730388 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9819d233-4251-4984-812c-3afe96372097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7391967dbca463b9b0a3ac3753cf96b', 'request_id': 'req-9819d233-4251-4984-812c-3afe96372097'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:50.738631 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9819d233-4251-4984-812c-3afe96372097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7391967dbca463b9b0a3ac3753cf96b', 'request_id': 'req-9819d233-4251-4984-812c-3afe96372097'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:50.744529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9819d233-4251-4984-812c-3afe96372097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7391967dbca463b9b0a3ac3753cf96b', 'request_id': 'req-9819d233-4251-4984-812c-3afe96372097'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:50.755519 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9819d233-4251-4984-812c-3afe96372097 admin admin] The token cache is being invalidate because project d7391967dbca463b9b0a3ac3753cf96b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:16:50.755806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9819d233-4251-4984-812c-3afe96372097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:50.756431 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-9819d233-4251-4984-812c-3afe96372097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:16:50.758067 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1264/2541] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:16:50 2022] DELETE /identity/v3/projects/d7391967dbca463b9b0a3ac3753cf96b => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:16:51.511964 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef5b28-2dfb-45e3-a378-6b6115291184 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:51.512530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef5b28-2dfb-45e3-a378-6b6115291184 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:51.512530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ef5b28-2dfb-45e3-a378-6b6115291184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:51.532274 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f0ef5b28-2dfb-45e3-a378-6b6115291184 None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:16:51.533495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f0ef5b28-2dfb-45e3-a378-6b6115291184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:51.558086 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:51.608530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1278/2542] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 19:16:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:16:51.621361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819d233-4251-4984-812c-3afe96372097 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:16:51.623414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9819d233-4251-4984-812c-3afe96372097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:16:51.641787 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:16:51.707886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9819d233-4251-4984-812c-3afe96372097 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:16:51.713526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4233135-4e5d-49a7-b5cc-f91137efc01d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:16:51.714142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4233135-4e5d-49a7-b5cc-f91137efc01d service swift] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:16:51.714423 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4233135-4e5d-49a7-b5cc-f91137efc01d service swift] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:16:51.714696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4233135-4e5d-49a7-b5cc-f91137efc01d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:16:51.723353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4233135-4e5d-49a7-b5cc-f91137efc01d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:16:51.725164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4233135-4e5d-49a7-b5cc-f91137efc01d service swift] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:16:51.736264 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1265/2543] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:16:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:17:01.599754 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=15 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:01.731530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=26 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:03.920626 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef5b28-2dfb-45e3-a378-6b6115291184 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:03.922192 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f0ef5b28-2dfb-45e3-a378-6b6115291184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:03.981127 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ef5b28-2dfb-45e3-a378-6b6115291184 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:03.984734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86b8c9-b91d-4801-be15-d999f33c5bdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:03.985373 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86b8c9-b91d-4801-be15-d999f33c5bdf service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:03.985657 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86b8c9-b91d-4801-be15-d999f33c5bdf service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:03.985931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86b8c9-b91d-4801-be15-d999f33c5bdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:03.987726 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4e86b8c9-b91d-4801-be15-d999f33c5bdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:04.059700 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86b8c9-b91d-4801-be15-d999f33c5bdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:04.061677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86b8c9-b91d-4801-be15-d999f33c5bdf service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:04.072861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1279/2544] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:07.806212 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4233135-4e5d-49a7-b5cc-f91137efc01d service swift] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:07.809002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f4233135-4e5d-49a7-b5cc-f91137efc01d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:07.868446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4233135-4e5d-49a7-b5cc-f91137efc01d service swift] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:07.872099 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab45e62-05d7-4751-b270-ef80da7cb21b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:07.872701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab45e62-05d7-4751-b270-ef80da7cb21b service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:07.872979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab45e62-05d7-4751-b270-ef80da7cb21b service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:07.873346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab45e62-05d7-4751-b270-ef80da7cb21b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:07.875250 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8ab45e62-05d7-4751-b270-ef80da7cb21b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:07.890032 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab45e62-05d7-4751-b270-ef80da7cb21b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:07.891472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab45e62-05d7-4751-b270-ef80da7cb21b service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:07.904418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1266/2545] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:14.067858 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=54 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:16.047839 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86b8c9-b91d-4801-be15-d999f33c5bdf service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:16.056480 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86b8c9-b91d-4801-be15-d999f33c5bdf service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:16.060118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea42b4d-5f4a-4a0a-b8dc-0b331cc48f79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:16.060782 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea42b4d-5f4a-4a0a-b8dc-0b331cc48f79 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:16.061133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea42b4d-5f4a-4a0a-b8dc-0b331cc48f79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:16.061443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea42b4d-5f4a-4a0a-b8dc-0b331cc48f79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:16.063648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bea42b4d-5f4a-4a0a-b8dc-0b331cc48f79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:16.126709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea42b4d-5f4a-4a0a-b8dc-0b331cc48f79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:16.128157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea42b4d-5f4a-4a0a-b8dc-0b331cc48f79 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:16.137393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1280/2546] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:16.926151 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bff1d-214c-4786-9957-d57371b123d9 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:16.927026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bff1d-214c-4786-9957-d57371b123d9 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:16.927321 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02bff1d-214c-4786-9957-d57371b123d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:16.948718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-c02bff1d-214c-4786-9957-d57371b123d9 None None] MFA Rules not processed for user `0bfa1533b0944bd681f4c2a33cf701b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:17:16.950085 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c02bff1d-214c-4786-9957-d57371b123d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:16.958552 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1267/2547] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:17:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:17:26.133230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=28 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:26.942202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=29 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:31.431528 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea42b4d-5f4a-4a0a-b8dc-0b331cc48f79 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:31.433011 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bea42b4d-5f4a-4a0a-b8dc-0b331cc48f79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:31.443922 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea42b4d-5f4a-4a0a-b8dc-0b331cc48f79 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:31.448969 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2cc2b1-9ae1-48c9-9b25-1db6a3f8e249 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:31.449595 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cc2b1-9ae1-48c9-9b25-1db6a3f8e249 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:31.449875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cc2b1-9ae1-48c9-9b25-1db6a3f8e249 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:31.450145 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2cc2b1-9ae1-48c9-9b25-1db6a3f8e249 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:31.451943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1f2cc2b1-9ae1-48c9-9b25-1db6a3f8e249 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:31.462051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2cc2b1-9ae1-48c9-9b25-1db6a3f8e249 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:31.463456 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2cc2b1-9ae1-48c9-9b25-1db6a3f8e249 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:31.472607 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1281/2548] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:41.468703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=6 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:45.538464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bff1d-214c-4786-9957-d57371b123d9 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:45.548058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02bff1d-214c-4786-9957-d57371b123d9 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:45.551774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f1d5a-e54f-4c00-8fde-8b9eeafd23c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:45.552366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f1d5a-e54f-4c00-8fde-8b9eeafd23c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:45.552641 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f1d5a-e54f-4c00-8fde-8b9eeafd23c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:45.552909 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5f1d5a-e54f-4c00-8fde-8b9eeafd23c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:45.554675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7b5f1d5a-e54f-4c00-8fde-8b9eeafd23c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:45.622784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f1d5a-e54f-4c00-8fde-8b9eeafd23c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:45.624255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5f1d5a-e54f-4c00-8fde-8b9eeafd23c3 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:45.634996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1268/2549] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:45 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:45.986665 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2cc2b1-9ae1-48c9-9b25-1db6a3f8e249 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:45.988875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1f2cc2b1-9ae1-48c9-9b25-1db6a3f8e249 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:46.005382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2cc2b1-9ae1-48c9-9b25-1db6a3f8e249 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:46.010991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853ef1a-7224-4683-89d0-94768420cdf7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:46.011589 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853ef1a-7224-4683-89d0-94768420cdf7 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:46.011868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853ef1a-7224-4683-89d0-94768420cdf7 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:46.012142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853ef1a-7224-4683-89d0-94768420cdf7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:46.013872 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3853ef1a-7224-4683-89d0-94768420cdf7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:46.028450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853ef1a-7224-4683-89d0-94768420cdf7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:46.030093 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853ef1a-7224-4683-89d0-94768420cdf7 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:46.042055 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1282/2550] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:17:47.555672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f1d5a-e54f-4c00-8fde-8b9eeafd23c3 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:17:47.557130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7b5f1d5a-e54f-4c00-8fde-8b9eeafd23c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:47.574037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5f1d5a-e54f-4c00-8fde-8b9eeafd23c3 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:17:47.580850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce7edb-846d-4fcc-882b-42ca3fd20830 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:17:47.581831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce7edb-846d-4fcc-882b-42ca3fd20830 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:17:47.582110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce7edb-846d-4fcc-882b-42ca3fd20830 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:17:47.582483 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce7edb-846d-4fcc-882b-42ca3fd20830 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:17:47.584755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4cce7edb-846d-4fcc-882b-42ca3fd20830 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:17:47.598586 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce7edb-846d-4fcc-882b-42ca3fd20830 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:17:47.600376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce7edb-846d-4fcc-882b-42ca3fd20830 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:17:47.612464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1269/2551] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:17:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:17:56.037558 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=6 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:17:57.608218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=36 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:06.843529 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dade0-6175-4412-8bc8-dede14ea5307 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:06.844066 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dade0-6175-4412-8bc8-dede14ea5307 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:06.844066 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dade0-6175-4412-8bc8-dede14ea5307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:06.862985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-6a9dade0-6175-4412-8bc8-dede14ea5307 None None] MFA Rules not processed for user `b1b8da543ad543bb95881450e4f9b585`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:18:06.864163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6a9dade0-6175-4412-8bc8-dede14ea5307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:06.917320 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1283/2552] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:18:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:18:07.151508 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce7edb-846d-4fcc-882b-42ca3fd20830 service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:07.160828 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce7edb-846d-4fcc-882b-42ca3fd20830 service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:07.165012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dd6a9-53a9-4f8c-b2e1-c6586c8a74f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:07.165644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dd6a9-53a9-4f8c-b2e1-c6586c8a74f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:07.165951 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dd6a9-53a9-4f8c-b2e1-c6586c8a74f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:07.166221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1dd6a9-53a9-4f8c-b2e1-c6586c8a74f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:07.167947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4f1dd6a9-53a9-4f8c-b2e1-c6586c8a74f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:07.177311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1dd6a9-53a9-4f8c-b2e1-c6586c8a74f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:07.178724 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1dd6a9-53a9-4f8c-b2e1-c6586c8a74f7 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:07.186526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1270/2553] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:07.503248 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dade0-6175-4412-8bc8-dede14ea5307 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:07.504686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6a9dade0-6175-4412-8bc8-dede14ea5307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:07.582922 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dade0-6175-4412-8bc8-dede14ea5307 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:07.589118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564af01-73f2-4f3f-ae68-8536a3c1a5f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:07.589782 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564af01-73f2-4f3f-ae68-8536a3c1a5f9 service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:07.590063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564af01-73f2-4f3f-ae68-8536a3c1a5f9 service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:07.590335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564af01-73f2-4f3f-ae68-8536a3c1a5f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:07.592332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a564af01-73f2-4f3f-ae68-8536a3c1a5f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:07.605052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564af01-73f2-4f3f-ae68-8536a3c1a5f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:07.606668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564af01-73f2-4f3f-ae68-8536a3c1a5f9 service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:07.617120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1284/2554] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:17.183073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=6 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:17.612613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=57 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:18:17.957713 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dd6a9-53a9-4f8c-b2e1-c6586c8a74f7 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:18:17.973892 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1dd6a9-53a9-4f8c-b2e1-c6586c8a74f7 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:18:17.981096 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1996f2-8161-442e-8174-e7977e437156 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:18:17.982787 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1996f2-8161-442e-8174-e7977e437156 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:18:17.983274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1996f2-8161-442e-8174-e7977e437156 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:18:17.983849 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1996f2-8161-442e-8174-e7977e437156 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:18:17.986107 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7b1996f2-8161-442e-8174-e7977e437156 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:18:18.000120 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1996f2-8161-442e-8174-e7977e437156 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:18:18.001838 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1996f2-8161-442e-8174-e7977e437156 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:18:18.012847 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1271/2555] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:18:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:18:28.009020 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=6 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:01.221333 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564af01-73f2-4f3f-ae68-8536a3c1a5f9 service placement] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:01.234637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564af01-73f2-4f3f-ae68-8536a3c1a5f9 service placement] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:01.240024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59821745-f097-4fb7-a60e-0559b76bdf80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:01.240615 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59821745-f097-4fb7-a60e-0559b76bdf80 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:01.240895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59821745-f097-4fb7-a60e-0559b76bdf80 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:01.241163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59821745-f097-4fb7-a60e-0559b76bdf80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:01.242890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-59821745-f097-4fb7-a60e-0559b76bdf80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:01.302678 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59821745-f097-4fb7-a60e-0559b76bdf80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:01.304248 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59821745-f097-4fb7-a60e-0559b76bdf80 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:01.314028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1285/2556] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:01 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:11.309660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=28 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:18.359209 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1996f2-8161-442e-8174-e7977e437156 service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:18.360635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7b1996f2-8161-442e-8174-e7977e437156 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:18.375129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1996f2-8161-442e-8174-e7977e437156 service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:18.380016 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38499153-cf19-4883-a1c5-2c72cbf80969 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:18.380607 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38499153-cf19-4883-a1c5-2c72cbf80969 service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:18.380880 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38499153-cf19-4883-a1c5-2c72cbf80969 service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:18.381172 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38499153-cf19-4883-a1c5-2c72cbf80969 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:18.382896 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-38499153-cf19-4883-a1c5-2c72cbf80969 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:18.396323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38499153-cf19-4883-a1c5-2c72cbf80969 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:18.397923 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38499153-cf19-4883-a1c5-2c72cbf80969 service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:18.408124 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1272/2557] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:18.425846 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59821745-f097-4fb7-a60e-0559b76bdf80 service nova] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:18.428006 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-59821745-f097-4fb7-a60e-0559b76bdf80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:18.502704 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59821745-f097-4fb7-a60e-0559b76bdf80 service nova] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:18.507074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca183b-0a6c-4bcc-825c-1e3ae5e3836a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:18.507686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca183b-0a6c-4bcc-825c-1e3ae5e3836a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:18.507961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca183b-0a6c-4bcc-825c-1e3ae5e3836a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:18.508230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca183b-0a6c-4bcc-825c-1e3ae5e3836a admin admin] PATH_INFO: `/v3/users/b1b8da543ad543bb95881450e4f9b585` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:18.509343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca183b-0a6c-4bcc-825c-1e3ae5e3836a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1b8da543ad543bb95881450e4f9b585)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:18.511287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca183b-0a6c-4bcc-825c-1e3ae5e3836a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:18.556852 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a9ca183b-0a6c-4bcc-825c-1e3ae5e3836a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1b8da543ad543bb95881450e4f9b585'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:18.563625 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a9ca183b-0a6c-4bcc-825c-1e3ae5e3836a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1b8da543ad543bb95881450e4f9b585'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:18.570775 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a9ca183b-0a6c-4bcc-825c-1e3ae5e3836a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1b8da543ad543bb95881450e4f9b585'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:18.571392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a9ca183b-0a6c-4bcc-825c-1e3ae5e3836a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1b8da543ad543bb95881450e4f9b585'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:18.583264 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a9ca183b-0a6c-4bcc-825c-1e3ae5e3836a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1b8da543ad543bb95881450e4f9b585'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:18.589470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1286/2558] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:19:18 2022] DELETE /identity/v3/users/b1b8da543ad543bb95881450e4f9b585 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:18.762332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38499153-cf19-4883-a1c5-2c72cbf80969 service placement] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:18.780142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38499153-cf19-4883-a1c5-2c72cbf80969 service placement] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:18.839231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f7c12-7673-40c7-bd98-85a5aeac38ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:18.839861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f7c12-7673-40c7-bd98-85a5aeac38ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:18.840132 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f7c12-7673-40c7-bd98-85a5aeac38ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:18.840404 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14f7c12-7673-40c7-bd98-85a5aeac38ac admin admin] PATH_INFO: `/v3/projects/08d5f61674804f93bd2127fc76919bd3` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:18.841556 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f7c12-7673-40c7-bd98-85a5aeac38ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=08d5f61674804f93bd2127fc76919bd3)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:18.843469 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14f7c12-7673-40c7-bd98-85a5aeac38ac admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:18.889830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d14f7c12-7673-40c7-bd98-85a5aeac38ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08d5f61674804f93bd2127fc76919bd3', 'request_id': 'req-d14f7c12-7673-40c7-bd98-85a5aeac38ac'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:18.897082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d14f7c12-7673-40c7-bd98-85a5aeac38ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08d5f61674804f93bd2127fc76919bd3', 'request_id': 'req-d14f7c12-7673-40c7-bd98-85a5aeac38ac'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:18.903481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d14f7c12-7673-40c7-bd98-85a5aeac38ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08d5f61674804f93bd2127fc76919bd3', 'request_id': 'req-d14f7c12-7673-40c7-bd98-85a5aeac38ac'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:18.914252 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d14f7c12-7673-40c7-bd98-85a5aeac38ac admin admin] The token cache is being invalidate because project 08d5f61674804f93bd2127fc76919bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:19:18.914536 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d14f7c12-7673-40c7-bd98-85a5aeac38ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:18.915168 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d14f7c12-7673-40c7-bd98-85a5aeac38ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:19:18.916713 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1273/2559] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:19:18 2022] DELETE /identity/v3/projects/08d5f61674804f93bd2127fc76919bd3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:19:28.585726 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:28.911292 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:30.633990 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7f32f-f8c7-47bd-ad04-200ac8d207b6 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:30.634533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7f32f-f8c7-47bd-ad04-200ac8d207b6 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:30.634533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a7f32f-f8c7-47bd-ad04-200ac8d207b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:30.655317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-66a7f32f-f8c7-47bd-ad04-200ac8d207b6 None None] MFA Rules not processed for user `aa81a686e4324370a20e3fc35e5cc7a5`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:30.656499 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-66a7f32f-f8c7-47bd-ad04-200ac8d207b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:30.710870 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1287/2560] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:19:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:38.756413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f7c12-7673-40c7-bd98-85a5aeac38ac admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:38.757866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d14f7c12-7673-40c7-bd98-85a5aeac38ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:38.831696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14f7c12-7673-40c7-bd98-85a5aeac38ac admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:38.837176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e67ec-0fc6-46d4-a74d-0007d457cc0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:38.837828 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e67ec-0fc6-46d4-a74d-0007d457cc0c service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:38.838104 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e67ec-0fc6-46d4-a74d-0007d457cc0c service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:38.838376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e67ec-0fc6-46d4-a74d-0007d457cc0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:38.840742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-749e67ec-0fc6-46d4-a74d-0007d457cc0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:38.900732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e67ec-0fc6-46d4-a74d-0007d457cc0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:38.902864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749e67ec-0fc6-46d4-a74d-0007d457cc0c service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:38.912279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1274/2561] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:40.704416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=25 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:48.908421 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=56 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:19:55.071546 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7f32f-f8c7-47bd-ad04-200ac8d207b6 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:55.073535 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-66a7f32f-f8c7-47bd-ad04-200ac8d207b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:55.129974 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a7f32f-f8c7-47bd-ad04-200ac8d207b6 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:55.133577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a458e-6fdc-419f-ba1e-812155455847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:55.134176 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a458e-6fdc-419f-ba1e-812155455847 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:55.134456 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a458e-6fdc-419f-ba1e-812155455847 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:55.134728 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a458e-6fdc-419f-ba1e-812155455847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:55.136997 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2d4a458e-6fdc-419f-ba1e-812155455847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:55.146508 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a458e-6fdc-419f-ba1e-812155455847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:55.147911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a458e-6fdc-419f-ba1e-812155455847 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:55.156427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1288/2562] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:55.278938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2506-5cec-4086-8ff5-54dac20ba8d4 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:55.279515 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2506-5cec-4086-8ff5-54dac20ba8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:55.279703 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daf2506-5cec-4086-8ff5-54dac20ba8d4 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:55.282035 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1275/2563] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 19:19:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:19:55.304448 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1f43d-1fc0-4da5-82f6-e9e84496660f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:55.304773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1f43d-1fc0-4da5-82f6-e9e84496660f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:55.305172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c1f43d-1fc0-4da5-82f6-e9e84496660f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:55.330339 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-61c1f43d-1fc0-4da5-82f6-e9e84496660f None None] MFA Rules not processed for user `093edc8c192c469f9532a45bb3326612`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:55.331726 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-61c1f43d-1fc0-4da5-82f6-e9e84496660f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:55.338594 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1289/2564] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 19:19:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:55.351725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf2506-5cec-4086-8ff5-54dac20ba8d4 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:55.364129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daf2506-5cec-4086-8ff5-54dac20ba8d4 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:55.369348 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a888851e-a61e-4c4e-967b-a7c96a527d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:55.370674 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888851e-a61e-4c4e-967b-a7c96a527d82 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:55.371117 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888851e-a61e-4c4e-967b-a7c96a527d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:55.371550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a888851e-a61e-4c4e-967b-a7c96a527d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:55.381658 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888851e-a61e-4c4e-967b-a7c96a527d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:55.383390 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a888851e-a61e-4c4e-967b-a7c96a527d82 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:55.392215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1276/2565] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:19:55.479186 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63fee3-1b36-4a6f-a118-69a7b274462a None None] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:55.479482 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63fee3-1b36-4a6f-a118-69a7b274462a None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:55.479755 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e63fee3-1b36-4a6f-a118-69a7b274462a None None] PATH_INFO: `/` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:55.481120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1290/2566] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 19:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 31 19:19:55.486673 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b701a-85be-4479-8249-b05a885325a4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:55.486959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b701a-85be-4479-8249-b05a885325a4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:55.487235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765b701a-85be-4479-8249-b05a885325a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:55.508217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-765b701a-85be-4479-8249-b05a885325a4 None None] MFA Rules not processed for user `673a72ac9a9847e3812007b66ac7a5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:19:55.509442 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-765b701a-85be-4479-8249-b05a885325a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:55.519343 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1277/2567] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 19:19:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:19:55.529923 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63fee3-1b36-4a6f-a118-69a7b274462a None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:19:55.531783 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7e63fee3-1b36-4a6f-a118-69a7b274462a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:19:55.545469 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e63fee3-1b36-4a6f-a118-69a7b274462a None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:19:55.549049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87095c4-fe26-420b-bc29-f6035625d917 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:19:55.549686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87095c4-fe26-420b-bc29-f6035625d917 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:19:55.549970 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87095c4-fe26-420b-bc29-f6035625d917 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:19:55.550272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87095c4-fe26-420b-bc29-f6035625d917 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:19:55.561569 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87095c4-fe26-420b-bc29-f6035625d917 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:19:55.562990 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87095c4-fe26-420b-bc29-f6035625d917 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:19:55.574838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1291/2568] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:19:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:20:05.214614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297499a-7a97-476b-963c-c05231f2f004 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:05.215148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297499a-7a97-476b-963c-c05231f2f004 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:05.215148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7297499a-7a97-476b-963c-c05231f2f004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:05.234183 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-7297499a-7a97-476b-963c-c05231f2f004 None None] MFA Rules not processed for user `0bfa1533b0944bd681f4c2a33cf701b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:20:05.235373 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7297499a-7a97-476b-963c-c05231f2f004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:05.289586 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1278/2569] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:20:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:20:05.569248 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=35 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:15.282327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=34 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:20:50.024314 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87095c4-fe26-420b-bc29-f6035625d917 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:50.025842 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e87095c4-fe26-420b-bc29-f6035625d917 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:50.168725 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87095c4-fe26-420b-bc29-f6035625d917 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:50.174748 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5366665-abb8-4367-8184-b1b1f4c3a5bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:50.175361 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5366665-abb8-4367-8184-b1b1f4c3a5bf service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:50.175636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5366665-abb8-4367-8184-b1b1f4c3a5bf service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:50.175905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5366665-abb8-4367-8184-b1b1f4c3a5bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:50.177650 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b5366665-abb8-4367-8184-b1b1f4c3a5bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:20:50.189162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5366665-abb8-4367-8184-b1b1f4c3a5bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:20:50.190787 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5366665-abb8-4367-8184-b1b1f4c3a5bf service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:20:50.201824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1292/2570] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:20:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:20:59.889095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297499a-7a97-476b-963c-c05231f2f004 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:20:59.911281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7297499a-7a97-476b-963c-c05231f2f004 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:20:59.914882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54496d0-f60b-404a-979e-ee64e6e1bd56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:20:59.915477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54496d0-f60b-404a-979e-ee64e6e1bd56 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:20:59.915756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54496d0-f60b-404a-979e-ee64e6e1bd56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:20:59.916030 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54496d0-f60b-404a-979e-ee64e6e1bd56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:20:59.917785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c54496d0-f60b-404a-979e-ee64e6e1bd56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:00.001277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54496d0-f60b-404a-979e-ee64e6e1bd56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:00.002658 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54496d0-f60b-404a-979e-ee64e6e1bd56 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:00.021318 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1279/2571] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:20:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:21:00.197830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=63 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:03.749499 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5366665-abb8-4367-8184-b1b1f4c3a5bf service placement] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:03.751542 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b5366665-abb8-4367-8184-b1b1f4c3a5bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:03.765530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5366665-abb8-4367-8184-b1b1f4c3a5bf service placement] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:03.770977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f93f35-406b-4973-bb0c-5f2cd5dcf97e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:03.771519 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f93f35-406b-4973-bb0c-5f2cd5dcf97e service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:03.771798 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f93f35-406b-4973-bb0c-5f2cd5dcf97e service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:03.772067 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f93f35-406b-4973-bb0c-5f2cd5dcf97e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:03.774022 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-07f93f35-406b-4973-bb0c-5f2cd5dcf97e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:03.789483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f93f35-406b-4973-bb0c-5f2cd5dcf97e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:03.791383 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f93f35-406b-4973-bb0c-5f2cd5dcf97e service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:03.803993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1293/2572] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:21:03.820560 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54496d0-f60b-404a-979e-ee64e6e1bd56 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:03.823542 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c54496d0-f60b-404a-979e-ee64e6e1bd56 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:03.837526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54496d0-f60b-404a-979e-ee64e6e1bd56 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:03.848406 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bbcea-4e1d-4f6f-9d1d-604c15db506d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:03.849139 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bbcea-4e1d-4f6f-9d1d-604c15db506d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:03.849373 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bbcea-4e1d-4f6f-9d1d-604c15db506d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:03.849644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bbcea-4e1d-4f6f-9d1d-604c15db506d admin admin] PATH_INFO: `/v3/users/aa81a686e4324370a20e3fc35e5cc7a5` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:03.850792 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bbcea-4e1d-4f6f-9d1d-604c15db506d admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa81a686e4324370a20e3fc35e5cc7a5)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:03.852756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bbcea-4e1d-4f6f-9d1d-604c15db506d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:03.904073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8f0bbcea-4e1d-4f6f-9d1d-604c15db506d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa81a686e4324370a20e3fc35e5cc7a5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:03.915518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8f0bbcea-4e1d-4f6f-9d1d-604c15db506d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa81a686e4324370a20e3fc35e5cc7a5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:03.930038 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8f0bbcea-4e1d-4f6f-9d1d-604c15db506d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa81a686e4324370a20e3fc35e5cc7a5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:03.931231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8f0bbcea-4e1d-4f6f-9d1d-604c15db506d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa81a686e4324370a20e3fc35e5cc7a5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:03.931895 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8f0bbcea-4e1d-4f6f-9d1d-604c15db506d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa81a686e4324370a20e3fc35e5cc7a5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:03.946472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1280/2573] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:21:03 2022] DELETE /identity/v3/users/aa81a686e4324370a20e3fc35e5cc7a5 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:04.165057 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f93f35-406b-4973-bb0c-5f2cd5dcf97e service placement] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:04.178347 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f93f35-406b-4973-bb0c-5f2cd5dcf97e service placement] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:04.241906 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644e970-2e2a-413d-9bb9-8c811fb1f03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:04.242531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644e970-2e2a-413d-9bb9-8c811fb1f03e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:04.242809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644e970-2e2a-413d-9bb9-8c811fb1f03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:04.243079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644e970-2e2a-413d-9bb9-8c811fb1f03e admin admin] PATH_INFO: `/v3/projects/e3aa913bb25c45039643ce87bb84cbc2` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:04.244185 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644e970-2e2a-413d-9bb9-8c811fb1f03e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3aa913bb25c45039643ce87bb84cbc2)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:04.246093 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644e970-2e2a-413d-9bb9-8c811fb1f03e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:04.296127 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2644e970-2e2a-413d-9bb9-8c811fb1f03e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3aa913bb25c45039643ce87bb84cbc2', 'request_id': 'req-2644e970-2e2a-413d-9bb9-8c811fb1f03e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:04.302137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2644e970-2e2a-413d-9bb9-8c811fb1f03e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3aa913bb25c45039643ce87bb84cbc2', 'request_id': 'req-2644e970-2e2a-413d-9bb9-8c811fb1f03e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:04.307833 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2644e970-2e2a-413d-9bb9-8c811fb1f03e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3aa913bb25c45039643ce87bb84cbc2', 'request_id': 'req-2644e970-2e2a-413d-9bb9-8c811fb1f03e'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:04.322639 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2644e970-2e2a-413d-9bb9-8c811fb1f03e admin admin] The token cache is being invalidate because project e3aa913bb25c45039643ce87bb84cbc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:21:04.322926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2644e970-2e2a-413d-9bb9-8c811fb1f03e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:04.323549 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2644e970-2e2a-413d-9bb9-8c811fb1f03e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:04.325028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1294/2574] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:21:04 2022] DELETE /identity/v3/projects/e3aa913bb25c45039643ce87bb84cbc2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:21:04.330054 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd678119-361c-4b31-b939-3247bb7df16c None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:04.330339 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd678119-361c-4b31-b939-3247bb7df16c None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:04.330631 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd678119-361c-4b31-b939-3247bb7df16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:04.378018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:04.414502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-bd678119-361c-4b31-b939-3247bb7df16c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:04.416722 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bd678119-361c-4b31-b939-3247bb7df16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:04.512290 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1281/2575] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:04.514521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf0ce1-f15d-4b66-ad51-f26065be6d3f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:04.514810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf0ce1-f15d-4b66-ad51-f26065be6d3f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:04.515084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf0ce1-f15d-4b66-ad51-f26065be6d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:04.536281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-bdcf0ce1-f15d-4b66-ad51-f26065be6d3f None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:04.537460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bdcf0ce1-f15d-4b66-ad51-f26065be6d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:04.548186 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1295/2576] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:04.552103 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd678119-361c-4b31-b939-3247bb7df16c None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:04.562504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd678119-361c-4b31-b939-3247bb7df16c None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:04.567464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f6de8-c161-4ae0-8733-2f3e4fb047e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:04.568137 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f6de8-c161-4ae0-8733-2f3e4fb047e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:04.568493 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f6de8-c161-4ae0-8733-2f3e4fb047e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:04.568975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2f6de8-c161-4ae0-8733-2f3e4fb047e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:04.569718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f6de8-c161-4ae0-8733-2f3e4fb047e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:21:04.570079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f6de8-c161-4ae0-8733-2f3e4fb047e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:04.571973 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2f6de8-c161-4ae0-8733-2f3e4fb047e9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:04.588681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1282/2577] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:21:04.590899 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf0ce1-f15d-4b66-ad51-f26065be6d3f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:04.605391 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf0ce1-f15d-4b66-ad51-f26065be6d3f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:04.612105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d2de8-48cd-42da-a478-c391f2c4f919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:04.613273 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d2de8-48cd-42da-a478-c391f2c4f919 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:04.613552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d2de8-48cd-42da-a478-c391f2c4f919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:04.613835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616d2de8-48cd-42da-a478-c391f2c4f919 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:04.614561 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d2de8-48cd-42da-a478-c391f2c4f919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:04.616452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616d2de8-48cd-42da-a478-c391f2c4f919 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:04.658451 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1296/2578] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:21:04 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:21:04.663146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f6de8-c161-4ae0-8733-2f3e4fb047e9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:04.676298 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2f6de8-c161-4ae0-8733-2f3e4fb047e9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:04.818657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84f05c-ba9f-400b-9128-08672408f674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:04.819650 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84f05c-ba9f-400b-9128-08672408f674 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:04.820186 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84f05c-ba9f-400b-9128-08672408f674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:04.820575 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84f05c-ba9f-400b-9128-08672408f674 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:04.821340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84f05c-ba9f-400b-9128-08672408f674 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:04.824567 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84f05c-ba9f-400b-9128-08672408f674 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:04.854408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1283/2579] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:21:04 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:21:04.859741 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d2de8-48cd-42da-a478-c391f2c4f919 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:04.874103 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616d2de8-48cd-42da-a478-c391f2c4f919 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:04.879068 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce9977-b239-49a6-9ab4-1940f0522946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:04.879991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce9977-b239-49a6-9ab4-1940f0522946 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:04.879991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce9977-b239-49a6-9ab4-1940f0522946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:04.880437 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce9977-b239-49a6-9ab4-1940f0522946 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:04.881246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce9977-b239-49a6-9ab4-1940f0522946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:04.883102 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce9977-b239-49a6-9ab4-1940f0522946 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:04.895028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1297/2580] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:21:04.899037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84f05c-ba9f-400b-9128-08672408f674 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:04.910663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84f05c-ba9f-400b-9128-08672408f674 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:04.915543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c8239-7d23-4ea7-b45f-90eeae6e7bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:04.916220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c8239-7d23-4ea7-b45f-90eeae6e7bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:04.916568 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c8239-7d23-4ea7-b45f-90eeae6e7bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:04.916942 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c8239-7d23-4ea7-b45f-90eeae6e7bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:04.917687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c8239-7d23-4ea7-b45f-90eeae6e7bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:04.920053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c8239-7d23-4ea7-b45f-90eeae6e7bb5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:04.934199 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1284/2581] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:21:04.938757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce9977-b239-49a6-9ab4-1940f0522946 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:04.949034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce9977-b239-49a6-9ab4-1940f0522946 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:04.953547 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f42994-3c3e-4752-b0c8-72e24b25c50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:04.954214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f42994-3c3e-4752-b0c8-72e24b25c50c admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:04.954466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f42994-3c3e-4752-b0c8-72e24b25c50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:04.954733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f42994-3c3e-4752-b0c8-72e24b25c50c admin admin] PATH_INFO: `/v3/projects/12835bbc76594286925e297e824e0d91/users/347993dd122543a684681c1a0783962c/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:04.970314 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f42994-3c3e-4752-b0c8-72e24b25c50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=12835bbc76594286925e297e824e0d91, user_id=347993dd122543a684681c1a0783962c, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:04.972937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f42994-3c3e-4752-b0c8-72e24b25c50c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:04.988510 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1298/2582] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:21:04 2022] PUT /identity/v3/projects/12835bbc76594286925e297e824e0d91/users/347993dd122543a684681c1a0783962c/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:04.993952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebccfa-1260-44f0-aa98-629c665018ef None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:04.994249 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebccfa-1260-44f0-aa98-629c665018ef None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:04.994519 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebccfa-1260-44f0-aa98-629c665018ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:05.013557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-2cebccfa-1260-44f0-aa98-629c665018ef None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:05.014840 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2cebccfa-1260-44f0-aa98-629c665018ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:05.067428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:05.077852 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1285/2583] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:05.087645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f42994-3c3e-4752-b0c8-72e24b25c50c admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:05.089474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-11f42994-3c3e-4752-b0c8-72e24b25c50c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:05.107116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:05.163117 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f42994-3c3e-4752-b0c8-72e24b25c50c admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:05.166678 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393f146-09f8-4f8b-b8ce-ea044143eb4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:05.167273 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393f146-09f8-4f8b-b8ce-ea044143eb4b service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:05.167546 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393f146-09f8-4f8b-b8ce-ea044143eb4b service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:05.167861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393f146-09f8-4f8b-b8ce-ea044143eb4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:05.178283 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393f146-09f8-4f8b-b8ce-ea044143eb4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:05.179736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393f146-09f8-4f8b-b8ce-ea044143eb4b service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:05.191198 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1299/2584] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:21:09.312417 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0310420-f278-402b-b1e4-1aed4a808eb4 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:09.312968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0310420-f278-402b-b1e4-1aed4a808eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:09.312968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0310420-f278-402b-b1e4-1aed4a808eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:09.346396 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b0310420-f278-402b-b1e4-1aed4a808eb4 None None] MFA Rules not processed for user `347993dd122543a684681c1a0783962c`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:09.347637 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b0310420-f278-402b-b1e4-1aed4a808eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:09.435548 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1286/2585] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:21:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:09.446636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393f146-09f8-4f8b-b8ce-ea044143eb4b service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:09.448249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f393f146-09f8-4f8b-b8ce-ea044143eb4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:09.572790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393f146-09f8-4f8b-b8ce-ea044143eb4b service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:09.580287 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f288e07-ce3b-4c49-ac66-4c04f3485ce4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:09.580907 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f288e07-ce3b-4c49-ac66-4c04f3485ce4 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:09.581211 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f288e07-ce3b-4c49-ac66-4c04f3485ce4 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:09.581484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f288e07-ce3b-4c49-ac66-4c04f3485ce4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:09.590842 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f288e07-ce3b-4c49-ac66-4c04f3485ce4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:09.592797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f288e07-ce3b-4c49-ac66-4c04f3485ce4 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:09.603495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1300/2586] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:21:11.072154 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cf19d-da94-4276-90fe-4720d62b307e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:11.073118 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cf19d-da94-4276-90fe-4720d62b307e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:11.073521 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cf19d-da94-4276-90fe-4720d62b307e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:11.107963 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ee6cf19d-da94-4276-90fe-4720d62b307e None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:11.109476 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ee6cf19d-da94-4276-90fe-4720d62b307e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:11.181044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1287/2587] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 31 19:21:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:11.183551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2106264-6ac7-41c0-a7db-2a20459e929f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:11.184250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2106264-6ac7-41c0-a7db-2a20459e929f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:11.185026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2106264-6ac7-41c0-a7db-2a20459e929f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:11.216409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f2106264-6ac7-41c0-a7db-2a20459e929f None None] MFA Rules not processed for user `347993dd122543a684681c1a0783962c`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:11.217722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f2106264-6ac7-41c0-a7db-2a20459e929f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:11.226326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1301/2588] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:21:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:11.242649 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6cf19d-da94-4276-90fe-4720d62b307e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:11.243992 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ee6cf19d-da94-4276-90fe-4720d62b307e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:11.325907 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6cf19d-da94-4276-90fe-4720d62b307e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:11.331904 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b90e8-f4cf-4d7f-b00a-8577f429abbe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:11.332700 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b90e8-f4cf-4d7f-b00a-8577f429abbe service swift] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:11.332999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b90e8-f4cf-4d7f-b00a-8577f429abbe service swift] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:11.333306 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797b90e8-f4cf-4d7f-b00a-8577f429abbe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:11.342546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b90e8-f4cf-4d7f-b00a-8577f429abbe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:11.344147 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797b90e8-f4cf-4d7f-b00a-8577f429abbe service swift] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:11.353060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1288/2589] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:21:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:21:13.564860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2106264-6ac7-41c0-a7db-2a20459e929f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:13.574919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2106264-6ac7-41c0-a7db-2a20459e929f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:13.581054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac761c2-824b-4128-a719-628a3341353a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:13.581929 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac761c2-824b-4128-a719-628a3341353a service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:13.582388 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac761c2-824b-4128-a719-628a3341353a service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:13.582807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac761c2-824b-4128-a719-628a3341353a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:13.584818 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-fac761c2-824b-4128-a719-628a3341353a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:13.602119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac761c2-824b-4128-a719-628a3341353a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:13.603580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac761c2-824b-4128-a719-628a3341353a service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:13.624442 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1302/2590] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:21:19.132018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b90e8-f4cf-4d7f-b00a-8577f429abbe service swift] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:19.133777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-797b90e8-f4cf-4d7f-b00a-8577f429abbe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:19.148177 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797b90e8-f4cf-4d7f-b00a-8577f429abbe service swift] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:19.155381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb201726-505d-4332-8398-0aea0f50cdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:19.156281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb201726-505d-4332-8398-0aea0f50cdc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:19.157067 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb201726-505d-4332-8398-0aea0f50cdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:19.157067 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb201726-505d-4332-8398-0aea0f50cdc9 admin admin] PATH_INFO: `/v3/users/a07fb9a216ac4683852fc6cae7c10b99` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:19.158344 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb201726-505d-4332-8398-0aea0f50cdc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a07fb9a216ac4683852fc6cae7c10b99)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:19.161299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb201726-505d-4332-8398-0aea0f50cdc9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:19.177635 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:19.254233 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eb201726-505d-4332-8398-0aea0f50cdc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a07fb9a216ac4683852fc6cae7c10b99'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.264535 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eb201726-505d-4332-8398-0aea0f50cdc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a07fb9a216ac4683852fc6cae7c10b99'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.278800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eb201726-505d-4332-8398-0aea0f50cdc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a07fb9a216ac4683852fc6cae7c10b99'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.279700 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eb201726-505d-4332-8398-0aea0f50cdc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a07fb9a216ac4683852fc6cae7c10b99'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.280547 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eb201726-505d-4332-8398-0aea0f50cdc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a07fb9a216ac4683852fc6cae7c10b99'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.296200 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1289/2591] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:21:19 2022] DELETE /identity/v3/users/a07fb9a216ac4683852fc6cae7c10b99 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:19.473695 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac761c2-824b-4128-a719-628a3341353a service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:19.491647 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac761c2-824b-4128-a719-628a3341353a service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:19.538807 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=100 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:19.555559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17e037-1a8f-453b-963f-c9fc97f5aa67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:19.556288 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17e037-1a8f-453b-963f-c9fc97f5aa67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:19.556633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17e037-1a8f-453b-963f-c9fc97f5aa67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:19.556972 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f17e037-1a8f-453b-963f-c9fc97f5aa67 admin admin] PATH_INFO: `/v3/projects/fe8ee0bdd09b45599a7c19798660aac0` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:19.558926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17e037-1a8f-453b-963f-c9fc97f5aa67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe8ee0bdd09b45599a7c19798660aac0)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:19.561074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f17e037-1a8f-453b-963f-c9fc97f5aa67 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:19.609768 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1f17e037-1a8f-453b-963f-c9fc97f5aa67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe8ee0bdd09b45599a7c19798660aac0', 'request_id': 'req-1f17e037-1a8f-453b-963f-c9fc97f5aa67'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.615792 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1f17e037-1a8f-453b-963f-c9fc97f5aa67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe8ee0bdd09b45599a7c19798660aac0', 'request_id': 'req-1f17e037-1a8f-453b-963f-c9fc97f5aa67'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.621299 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1f17e037-1a8f-453b-963f-c9fc97f5aa67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe8ee0bdd09b45599a7c19798660aac0', 'request_id': 'req-1f17e037-1a8f-453b-963f-c9fc97f5aa67'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.633130 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1f17e037-1a8f-453b-963f-c9fc97f5aa67 admin admin] The token cache is being invalidate because project fe8ee0bdd09b45599a7c19798660aac0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:21:19.633463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1f17e037-1a8f-453b-963f-c9fc97f5aa67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.634071 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1f17e037-1a8f-453b-963f-c9fc97f5aa67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.635530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1303/2592] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:21:19 2022] DELETE /identity/v3/projects/fe8ee0bdd09b45599a7c19798660aac0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:19.640076 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb201726-505d-4332-8398-0aea0f50cdc9 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:19.641519 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-eb201726-505d-4332-8398-0aea0f50cdc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:19.654990 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb201726-505d-4332-8398-0aea0f50cdc9 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:19.714398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d934940-5c30-4601-a612-5f8280431290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:19.715012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d934940-5c30-4601-a612-5f8280431290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:19.715291 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d934940-5c30-4601-a612-5f8280431290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:19.716172 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d934940-5c30-4601-a612-5f8280431290 admin admin] PATH_INFO: `/v3/users/e3c28a6f05274f10af0944d6d209f098` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:19.717757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d934940-5c30-4601-a612-5f8280431290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3c28a6f05274f10af0944d6d209f098)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:19.719650 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d934940-5c30-4601-a612-5f8280431290 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:19.772836 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2d934940-5c30-4601-a612-5f8280431290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3c28a6f05274f10af0944d6d209f098'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.780703 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2d934940-5c30-4601-a612-5f8280431290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3c28a6f05274f10af0944d6d209f098'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.792819 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2d934940-5c30-4601-a612-5f8280431290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3c28a6f05274f10af0944d6d209f098'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.793598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2d934940-5c30-4601-a612-5f8280431290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3c28a6f05274f10af0944d6d209f098'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.794180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2d934940-5c30-4601-a612-5f8280431290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3c28a6f05274f10af0944d6d209f098'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:19.806771 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1290/2593] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:21:19 2022] DELETE /identity/v3/users/e3c28a6f05274f10af0944d6d209f098 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:19.979730 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17e037-1a8f-453b-963f-c9fc97f5aa67 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:19.981528 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1f17e037-1a8f-453b-963f-c9fc97f5aa67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:19.994696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f17e037-1a8f-453b-963f-c9fc97f5aa67 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:20.052460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a99fd-0141-49bc-98a2-d03b41a22e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:20.053081 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a99fd-0141-49bc-98a2-d03b41a22e49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:20.053384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a99fd-0141-49bc-98a2-d03b41a22e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:20.053659 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664a99fd-0141-49bc-98a2-d03b41a22e49 admin admin] PATH_INFO: `/v3/projects/1c877d864d984a3eb0a752a6d9c4e697` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:20.054753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a99fd-0141-49bc-98a2-d03b41a22e49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c877d864d984a3eb0a752a6d9c4e697)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:20.056606 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664a99fd-0141-49bc-98a2-d03b41a22e49 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:20.105814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-664a99fd-0141-49bc-98a2-d03b41a22e49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c877d864d984a3eb0a752a6d9c4e697', 'request_id': 'req-664a99fd-0141-49bc-98a2-d03b41a22e49'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:20.112112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-664a99fd-0141-49bc-98a2-d03b41a22e49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c877d864d984a3eb0a752a6d9c4e697', 'request_id': 'req-664a99fd-0141-49bc-98a2-d03b41a22e49'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:20.117698 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-664a99fd-0141-49bc-98a2-d03b41a22e49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c877d864d984a3eb0a752a6d9c4e697', 'request_id': 'req-664a99fd-0141-49bc-98a2-d03b41a22e49'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:20.130586 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-664a99fd-0141-49bc-98a2-d03b41a22e49 admin admin] The token cache is being invalidate because project 1c877d864d984a3eb0a752a6d9c4e697 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:21:20.130863 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-664a99fd-0141-49bc-98a2-d03b41a22e49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:20.131463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-664a99fd-0141-49bc-98a2-d03b41a22e49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:21:20.132952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1304/2594] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:21:19 2022] DELETE /identity/v3/projects/1c877d864d984a3eb0a752a6d9c4e697 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:21:29.803090 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=20,keystone:SELECT=75,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:30.127603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=82,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:21:51.557591 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbc004-3aa6-43fc-bf8a-878e3e6c6d7f None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:51.558165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbc004-3aa6-43fc-bf8a-878e3e6c6d7f None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:51.558518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bbc004-3aa6-43fc-bf8a-878e3e6c6d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:51.580394 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-53bbc004-3aa6-43fc-bf8a-878e3e6c6d7f None None] MFA Rules not processed for user `0bfa1533b0944bd681f4c2a33cf701b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:21:51.581831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-53bbc004-3aa6-43fc-bf8a-878e3e6c6d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:51.656817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1291/2595] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:21:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:21:56.684438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a99fd-0141-49bc-98a2-d03b41a22e49 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:21:56.685865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-664a99fd-0141-49bc-98a2-d03b41a22e49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:56.743373 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664a99fd-0141-49bc-98a2-d03b41a22e49 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:21:56.747019 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc2b89-452d-4cac-8a65-b2be9c8972ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:21:56.747618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc2b89-452d-4cac-8a65-b2be9c8972ca service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:21:56.747895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc2b89-452d-4cac-8a65-b2be9c8972ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:21:56.748162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc2b89-452d-4cac-8a65-b2be9c8972ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:21:56.749889 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f7fc2b89-452d-4cac-8a65-b2be9c8972ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:21:56.811055 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc2b89-452d-4cac-8a65-b2be9c8972ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:21:56.812468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc2b89-452d-4cac-8a65-b2be9c8972ca service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:21:56.821922 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1305/2596] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:21:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:22:01.642338 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=25 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:22:03.472895 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbc004-3aa6-43fc-bf8a-878e3e6c6d7f None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:03.474448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-53bbc004-3aa6-43fc-bf8a-878e3e6c6d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:03.492196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bbc004-3aa6-43fc-bf8a-878e3e6c6d7f None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:03.499116 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24d978-9d6f-4dd1-89ab-2cb89bec6f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:03.500008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24d978-9d6f-4dd1-89ab-2cb89bec6f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:03.500465 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24d978-9d6f-4dd1-89ab-2cb89bec6f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:03.500907 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd24d978-9d6f-4dd1-89ab-2cb89bec6f7e admin admin] PATH_INFO: `/v3/users/26cd75e2c118412a88bb354e18863d34` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:03.502514 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24d978-9d6f-4dd1-89ab-2cb89bec6f7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=26cd75e2c118412a88bb354e18863d34)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:03.505681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd24d978-9d6f-4dd1-89ab-2cb89bec6f7e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:03.556613 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bd24d978-9d6f-4dd1-89ab-2cb89bec6f7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26cd75e2c118412a88bb354e18863d34'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:03.563942 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bd24d978-9d6f-4dd1-89ab-2cb89bec6f7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26cd75e2c118412a88bb354e18863d34'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:03.576446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bd24d978-9d6f-4dd1-89ab-2cb89bec6f7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26cd75e2c118412a88bb354e18863d34'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:03.577165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bd24d978-9d6f-4dd1-89ab-2cb89bec6f7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26cd75e2c118412a88bb354e18863d34'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:03.577892 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-bd24d978-9d6f-4dd1-89ab-2cb89bec6f7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26cd75e2c118412a88bb354e18863d34'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:03.592074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1292/2597] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:22:03 2022] DELETE /identity/v3/users/26cd75e2c118412a88bb354e18863d34 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:22:03.796984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc2b89-452d-4cac-8a65-b2be9c8972ca service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:03.807423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc2b89-452d-4cac-8a65-b2be9c8972ca service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:03.864311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcebbe4-d5de-4180-b764-5029d5336925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:03.864954 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcebbe4-d5de-4180-b764-5029d5336925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:03.865252 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcebbe4-d5de-4180-b764-5029d5336925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:03.865522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcebbe4-d5de-4180-b764-5029d5336925 admin admin] PATH_INFO: `/v3/projects/fe3b687e68734a8888291f0c3527acbc` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:03.866607 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcebbe4-d5de-4180-b764-5029d5336925 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe3b687e68734a8888291f0c3527acbc)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:03.868474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcebbe4-d5de-4180-b764-5029d5336925 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:03.913447 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-edcebbe4-d5de-4180-b764-5029d5336925 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe3b687e68734a8888291f0c3527acbc', 'request_id': 'req-edcebbe4-d5de-4180-b764-5029d5336925'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:03.915291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:22:03.920307 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-edcebbe4-d5de-4180-b764-5029d5336925 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe3b687e68734a8888291f0c3527acbc', 'request_id': 'req-edcebbe4-d5de-4180-b764-5029d5336925'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:03.926329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-edcebbe4-d5de-4180-b764-5029d5336925 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe3b687e68734a8888291f0c3527acbc', 'request_id': 'req-edcebbe4-d5de-4180-b764-5029d5336925'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:03.937993 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-edcebbe4-d5de-4180-b764-5029d5336925 admin admin] The token cache is being invalidate because project fe3b687e68734a8888291f0c3527acbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:22:03.938275 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-edcebbe4-d5de-4180-b764-5029d5336925 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:03.938888 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-edcebbe4-d5de-4180-b764-5029d5336925 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:03.940324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1306/2598] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:22:03 2022] DELETE /identity/v3/projects/fe3b687e68734a8888291f0c3527acbc => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:22:03.944420 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24d978-9d6f-4dd1-89ab-2cb89bec6f7e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:03.945890 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bd24d978-9d6f-4dd1-89ab-2cb89bec6f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:03.959962 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd24d978-9d6f-4dd1-89ab-2cb89bec6f7e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:04.016414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab6b52-40ae-4c26-a047-b9b2b61884a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:04.017037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab6b52-40ae-4c26-a047-b9b2b61884a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:04.017344 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab6b52-40ae-4c26-a047-b9b2b61884a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:04.017614 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab6b52-40ae-4c26-a047-b9b2b61884a7 admin admin] PATH_INFO: `/v3/users/eb940cd2e00c455c947e0bc0ad4c4d97` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:04.018730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab6b52-40ae-4c26-a047-b9b2b61884a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb940cd2e00c455c947e0bc0ad4c4d97)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:04.020624 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab6b52-40ae-4c26-a047-b9b2b61884a7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:04.064777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-23ab6b52-40ae-4c26-a047-b9b2b61884a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb940cd2e00c455c947e0bc0ad4c4d97'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:04.071543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-23ab6b52-40ae-4c26-a047-b9b2b61884a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb940cd2e00c455c947e0bc0ad4c4d97'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:04.083776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-23ab6b52-40ae-4c26-a047-b9b2b61884a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb940cd2e00c455c947e0bc0ad4c4d97'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:04.084463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-23ab6b52-40ae-4c26-a047-b9b2b61884a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb940cd2e00c455c947e0bc0ad4c4d97'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:04.085069 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-23ab6b52-40ae-4c26-a047-b9b2b61884a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb940cd2e00c455c947e0bc0ad4c4d97'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:04.086744 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:22:04.099680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1293/2599] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:22:03 2022] DELETE /identity/v3/users/eb940cd2e00c455c947e0bc0ad4c4d97 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:22:04.286469 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcebbe4-d5de-4180-b764-5029d5336925 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:04.287970 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-edcebbe4-d5de-4180-b764-5029d5336925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:04.303094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcebbe4-d5de-4180-b764-5029d5336925 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:04.362084 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2233486-fab7-411d-868a-d163528070b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:04.362709 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2233486-fab7-411d-868a-d163528070b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:04.363024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2233486-fab7-411d-868a-d163528070b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:04.363288 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2233486-fab7-411d-868a-d163528070b5 admin admin] PATH_INFO: `/v3/projects/b19d4a8c14834bc79ebedc8496a1982d` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:04.364393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2233486-fab7-411d-868a-d163528070b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b19d4a8c14834bc79ebedc8496a1982d)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:04.366327 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2233486-fab7-411d-868a-d163528070b5 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:04.411202 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c2233486-fab7-411d-868a-d163528070b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b19d4a8c14834bc79ebedc8496a1982d', 'request_id': 'req-c2233486-fab7-411d-868a-d163528070b5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:04.417127 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c2233486-fab7-411d-868a-d163528070b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b19d4a8c14834bc79ebedc8496a1982d', 'request_id': 'req-c2233486-fab7-411d-868a-d163528070b5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:04.423172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c2233486-fab7-411d-868a-d163528070b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b19d4a8c14834bc79ebedc8496a1982d', 'request_id': 'req-c2233486-fab7-411d-868a-d163528070b5'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:04.435629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c2233486-fab7-411d-868a-d163528070b5 admin admin] The token cache is being invalidate because project b19d4a8c14834bc79ebedc8496a1982d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:22:04.435911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c2233486-fab7-411d-868a-d163528070b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:04.436524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-c2233486-fab7-411d-868a-d163528070b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:22:04.438070 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1307/2600] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:22:04 2022] DELETE /identity/v3/projects/b19d4a8c14834bc79ebedc8496a1982d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:22:05.782126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab6b52-40ae-4c26-a047-b9b2b61884a7 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:05.783573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-23ab6b52-40ae-4c26-a047-b9b2b61884a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:05.838764 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab6b52-40ae-4c26-a047-b9b2b61884a7 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:05.885702 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d12902-46e8-4eb0-b4d1-ce75d47eca6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:05.886304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d12902-46e8-4eb0-b4d1-ce75d47eca6b service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:05.886580 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d12902-46e8-4eb0-b4d1-ce75d47eca6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:05.886840 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d12902-46e8-4eb0-b4d1-ce75d47eca6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:05.888531 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-03d12902-46e8-4eb0-b4d1-ce75d47eca6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:05.955967 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d12902-46e8-4eb0-b4d1-ce75d47eca6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:05.957401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d12902-46e8-4eb0-b4d1-ce75d47eca6b service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:05.968564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1294/2601] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:22:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:22:14.432798 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=56,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:22:15.963254 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=76 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:22:41.195614 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2233486-fab7-411d-868a-d163528070b5 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:41.197427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c2233486-fab7-411d-868a-d163528070b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:41.209929 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2233486-fab7-411d-868a-d163528070b5 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:41.213661 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46661645-82a8-439d-8a41-cf389449e940 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:41.214347 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46661645-82a8-439d-8a41-cf389449e940 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:41.214630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46661645-82a8-439d-8a41-cf389449e940 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:41.214904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46661645-82a8-439d-8a41-cf389449e940 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:41.216625 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-46661645-82a8-439d-8a41-cf389449e940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:41.232618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46661645-82a8-439d-8a41-cf389449e940 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:41.234139 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46661645-82a8-439d-8a41-cf389449e940 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:41.245091 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1308/2602] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:22:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:22:51.240297 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=6 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:22:59.310340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05853380-1620-4c2b-a22f-b33be5ed1c70 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:59.311081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05853380-1620-4c2b-a22f-b33be5ed1c70 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:59.311081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05853380-1620-4c2b-a22f-b33be5ed1c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:59.361094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-05853380-1620-4c2b-a22f-b33be5ed1c70 None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:22:59.363399 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-05853380-1620-4c2b-a22f-b33be5ed1c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:59.425043 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1295/2603] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 19:22:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:22:59.471055 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46661645-82a8-439d-8a41-cf389449e940 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:22:59.472441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-46661645-82a8-439d-8a41-cf389449e940 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:22:59.562283 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46661645-82a8-439d-8a41-cf389449e940 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:22:59.567142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d52558-54a6-4a08-a18c-8093eaab5af4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:22:59.567733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d52558-54a6-4a08-a18c-8093eaab5af4 service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:22:59.568013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d52558-54a6-4a08-a18c-8093eaab5af4 service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:22:59.568286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d52558-54a6-4a08-a18c-8093eaab5af4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:22:59.576864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d52558-54a6-4a08-a18c-8093eaab5af4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:22:59.578459 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d52558-54a6-4a08-a18c-8093eaab5af4 service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:22:59.586327 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1309/2604] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:22:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 31 19:23:09.422248 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=40 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:09.559259 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05853380-1620-4c2b-a22f-b33be5ed1c70 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:09.561367 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-05853380-1620-4c2b-a22f-b33be5ed1c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:09.572714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05853380-1620-4c2b-a22f-b33be5ed1c70 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:09.576277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b30174-8a3b-403b-b7c4-eb56c9f94b24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:09.576866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b30174-8a3b-403b-b7c4-eb56c9f94b24 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:09.577145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b30174-8a3b-403b-b7c4-eb56c9f94b24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:09.577452 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b30174-8a3b-403b-b7c4-eb56c9f94b24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:09.579091 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-14b30174-8a3b-403b-b7c4-eb56c9f94b24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:09.583139 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=41 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:09.588599 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b30174-8a3b-403b-b7c4-eb56c9f94b24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:09.590045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b30174-8a3b-403b-b7c4-eb56c9f94b24 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:09.598272 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1296/2605] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:13.022563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d52558-54a6-4a08-a18c-8093eaab5af4 service swift] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:13.032269 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d52558-54a6-4a08-a18c-8093eaab5af4 service swift] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:13.036028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83546e19-4e44-4bac-883f-8b325601b0fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:13.036607 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83546e19-4e44-4bac-883f-8b325601b0fb service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:13.036879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83546e19-4e44-4bac-883f-8b325601b0fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:13.037141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83546e19-4e44-4bac-883f-8b325601b0fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:13.038830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-83546e19-4e44-4bac-883f-8b325601b0fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:13.048848 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83546e19-4e44-4bac-883f-8b325601b0fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:13.050274 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83546e19-4e44-4bac-883f-8b325601b0fb service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:13.058411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1310/2606] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:13 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:13.179866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799fd18-97cd-4080-afe7-db4f2914cbc5 None None] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:13.180401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799fd18-97cd-4080-afe7-db4f2914cbc5 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:13.180401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9799fd18-97cd-4080-afe7-db4f2914cbc5 None None] PATH_INFO: `/` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:13.181847 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1297/2607] 10.222.0.65 () {54 vars in 921 bytes} [Wed Aug 31 19:23:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 31 19:23:13.187408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78455b-a03b-4e4d-bd7f-7c6922fd7609 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:13.187696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78455b-a03b-4e4d-bd7f-7c6922fd7609 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:13.187985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c78455b-a03b-4e4d-bd7f-7c6922fd7609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:13.231732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4c78455b-a03b-4e4d-bd7f-7c6922fd7609 None None] MFA Rules not processed for user `673a72ac9a9847e3812007b66ac7a5ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:13.232971 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4c78455b-a03b-4e4d-bd7f-7c6922fd7609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:13.242381 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1311/2608] 10.222.0.65 () {58 vars in 1018 bytes} [Wed Aug 31 19:23:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:23:13.254980 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799fd18-97cd-4080-afe7-db4f2914cbc5 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:13.256663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9799fd18-97cd-4080-afe7-db4f2914cbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:13.341774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9799fd18-97cd-4080-afe7-db4f2914cbc5 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:13.345904 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9b3c62-49c5-433a-9b82-ca013140ffa6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1863336fecad4987acdfd0b3f7b2f70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:13.346543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b3c62-49c5-433a-9b82-ca013140ffa6 service glance] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:13.346818 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b3c62-49c5-433a-9b82-ca013140ffa6 service glance] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:13.347087 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9b3c62-49c5-433a-9b82-ca013140ffa6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:13.358892 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b3c62-49c5-433a-9b82-ca013140ffa6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:13.360525 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9b3c62-49c5-433a-9b82-ca013140ffa6 service glance] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:13.372051 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1298/2609] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:13.571209 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78455b-a03b-4e4d-bd7f-7c6922fd7609 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:13.572581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4c78455b-a03b-4e4d-bd7f-7c6922fd7609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:13.643657 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c78455b-a03b-4e4d-bd7f-7c6922fd7609 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:13.650951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f905ea8-0f60-4fe3-966f-31800fab6b0d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:13.651956 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f905ea8-0f60-4fe3-966f-31800fab6b0d service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:13.652443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f905ea8-0f60-4fe3-966f-31800fab6b0d service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:13.652730 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f905ea8-0f60-4fe3-966f-31800fab6b0d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:13.655063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7f905ea8-0f60-4fe3-966f-31800fab6b0d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:13.667105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f905ea8-0f60-4fe3-966f-31800fab6b0d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:13.671038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f905ea8-0f60-4fe3-966f-31800fab6b0d service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:13.681696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1312/2610] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:16.170957 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9cb92-5b8d-45d2-a4f9-4612ef0744c3 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:16.171435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9cb92-5b8d-45d2-a4f9-4612ef0744c3 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:16.171532 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d9cb92-5b8d-45d2-a4f9-4612ef0744c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:16.190602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-03d9cb92-5b8d-45d2-a4f9-4612ef0744c3 None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:16.192539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-03d9cb92-5b8d-45d2-a4f9-4612ef0744c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:16.205580 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1299/2611] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 19:23:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:16.219035 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f905ea8-0f60-4fe3-966f-31800fab6b0d service placement] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:16.230635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f905ea8-0f60-4fe3-966f-31800fab6b0d service placement] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:16.235439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adfc1a7-5993-41ab-9018-fafcb799f170 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:16.236024 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfc1a7-5993-41ab-9018-fafcb799f170 service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:16.236303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfc1a7-5993-41ab-9018-fafcb799f170 service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:16.236577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adfc1a7-5993-41ab-9018-fafcb799f170 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:16.244357 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adfc1a7-5993-41ab-9018-fafcb799f170 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:16.245943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adfc1a7-5993-41ab-9018-fafcb799f170 service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:16.253767 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1313/2612] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:23:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:23:22.173054 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9cb92-5b8d-45d2-a4f9-4612ef0744c3 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:22.174593 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-03d9cb92-5b8d-45d2-a4f9-4612ef0744c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:22.191617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d9cb92-5b8d-45d2-a4f9-4612ef0744c3 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:22.197484 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a21419-624a-42e9-8802-ca9b1e1cd57a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:22.198174 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a21419-624a-42e9-8802-ca9b1e1cd57a service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:22.198520 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a21419-624a-42e9-8802-ca9b1e1cd57a service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:22.198858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a21419-624a-42e9-8802-ca9b1e1cd57a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:22.201210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d4a21419-624a-42e9-8802-ca9b1e1cd57a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:22.292136 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a21419-624a-42e9-8802-ca9b1e1cd57a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:22.293788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a21419-624a-42e9-8802-ca9b1e1cd57a service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:22.314062 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1300/2613] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:26.250647 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=53 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:32.311255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=85 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:41.577342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adfc1a7-5993-41ab-9018-fafcb799f170 service swift] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:41.590856 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adfc1a7-5993-41ab-9018-fafcb799f170 service swift] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:41.596947 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80fb381-c039-4a5d-9ce8-32fa134f213c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:41.597691 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fb381-c039-4a5d-9ce8-32fa134f213c service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:41.598080 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fb381-c039-4a5d-9ce8-32fa134f213c service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:41.598500 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80fb381-c039-4a5d-9ce8-32fa134f213c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:41.600705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d80fb381-c039-4a5d-9ce8-32fa134f213c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:41.633141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fb381-c039-4a5d-9ce8-32fa134f213c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:41.637588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80fb381-c039-4a5d-9ce8-32fa134f213c service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:41.664608 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1314/2614] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:42.148591 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a21419-624a-42e9-8802-ca9b1e1cd57a service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:42.150275 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d4a21419-624a-42e9-8802-ca9b1e1cd57a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:42.168805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a21419-624a-42e9-8802-ca9b1e1cd57a service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:42.175494 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ad246-ba6b-483a-a1f6-b2214e8915cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:42.176403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ad246-ba6b-483a-a1f6-b2214e8915cf service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:42.176847 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ad246-ba6b-483a-a1f6-b2214e8915cf service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:42.177307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ad246-ba6b-483a-a1f6-b2214e8915cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:42.181131 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-fe0ad246-ba6b-483a-a1f6-b2214e8915cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:42.200670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ad246-ba6b-483a-a1f6-b2214e8915cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:42.206388 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0ad246-ba6b-483a-a1f6-b2214e8915cf service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:42.222060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1301/2615] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:44.079052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ee39c-5ed1-4b19-a470-5a9930e75cf3 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:44.079566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ee39c-5ed1-4b19-a470-5a9930e75cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:44.079668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ee39c-5ed1-4b19-a470-5a9930e75cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:44.101588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a82ee39c-5ed1-4b19-a470-5a9930e75cf3 None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:44.102830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a82ee39c-5ed1-4b19-a470-5a9930e75cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:44.114511 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1315/2616] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 31 19:23:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:44.130891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ad246-ba6b-483a-a1f6-b2214e8915cf service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:44.145316 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ad246-ba6b-483a-a1f6-b2214e8915cf service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:44.151165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a5cdc-4020-4c88-b83c-705fbe139a7b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:44.151687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a5cdc-4020-4c88-b83c-705fbe139a7b service swift] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:44.151952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a5cdc-4020-4c88-b83c-705fbe139a7b service swift] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:44.152236 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a5cdc-4020-4c88-b83c-705fbe139a7b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:44.161537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a5cdc-4020-4c88-b83c-705fbe139a7b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:44.163104 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429a5cdc-4020-4c88-b83c-705fbe139a7b service swift] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:44.173869 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1302/2617] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:23:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:23:48.382875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ee39c-5ed1-4b19-a470-5a9930e75cf3 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:48.392886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ee39c-5ed1-4b19-a470-5a9930e75cf3 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:48.396505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c5c7f-321f-4d55-985a-fd2deacced10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:48.397130 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c5c7f-321f-4d55-985a-fd2deacced10 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:48.397436 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c5c7f-321f-4d55-985a-fd2deacced10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:48.397705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c5c7f-321f-4d55-985a-fd2deacced10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:48.399412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f30c5c7f-321f-4d55-985a-fd2deacced10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:48.462315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c5c7f-321f-4d55-985a-fd2deacced10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:48.463655 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c5c7f-321f-4d55-985a-fd2deacced10 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:48.474243 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1316/2618] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:51.583838 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a5cdc-4020-4c88-b83c-705fbe139a7b service swift] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:51.585332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-429a5cdc-4020-4c88-b83c-705fbe139a7b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:51.600124 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a5cdc-4020-4c88-b83c-705fbe139a7b service swift] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:51.604603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a3e8c1-790c-44ce-9beb-65422aa83791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:51.605257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3e8c1-790c-44ce-9beb-65422aa83791 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:51.605539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3e8c1-790c-44ce-9beb-65422aa83791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:51.605927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a3e8c1-790c-44ce-9beb-65422aa83791 admin admin] PATH_INFO: `/v3/users/0bfa1533b0944bd681f4c2a33cf701b0` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:51.607053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3e8c1-790c-44ce-9beb-65422aa83791 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bfa1533b0944bd681f4c2a33cf701b0)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:51.608946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a3e8c1-790c-44ce-9beb-65422aa83791 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:51.656447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d2a3e8c1-790c-44ce-9beb-65422aa83791 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bfa1533b0944bd681f4c2a33cf701b0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:51.664047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d2a3e8c1-790c-44ce-9beb-65422aa83791 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bfa1533b0944bd681f4c2a33cf701b0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:51.677154 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d2a3e8c1-790c-44ce-9beb-65422aa83791 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bfa1533b0944bd681f4c2a33cf701b0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:51.677870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d2a3e8c1-790c-44ce-9beb-65422aa83791 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bfa1533b0944bd681f4c2a33cf701b0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:51.678502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d2a3e8c1-790c-44ce-9beb-65422aa83791 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bfa1533b0944bd681f4c2a33cf701b0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:51.690492 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1303/2619] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:23:51 2022] DELETE /identity/v3/users/0bfa1533b0944bd681f4c2a33cf701b0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:51.863396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c5c7f-321f-4d55-985a-fd2deacced10 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:51.873726 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c5c7f-321f-4d55-985a-fd2deacced10 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:51.929902 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f12b55-5ff6-4aeb-86eb-7d9f8417f9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:51.930511 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f12b55-5ff6-4aeb-86eb-7d9f8417f9b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:51.930836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f12b55-5ff6-4aeb-86eb-7d9f8417f9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:51.931050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f12b55-5ff6-4aeb-86eb-7d9f8417f9b6 admin admin] PATH_INFO: `/v3/projects/e6d782400f1e432c8d10a31334c42e68` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:51.932151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f12b55-5ff6-4aeb-86eb-7d9f8417f9b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6d782400f1e432c8d10a31334c42e68)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:51.934041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f12b55-5ff6-4aeb-86eb-7d9f8417f9b6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:51.980682 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d9f12b55-5ff6-4aeb-86eb-7d9f8417f9b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6d782400f1e432c8d10a31334c42e68', 'request_id': 'req-d9f12b55-5ff6-4aeb-86eb-7d9f8417f9b6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:51.986564 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d9f12b55-5ff6-4aeb-86eb-7d9f8417f9b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6d782400f1e432c8d10a31334c42e68', 'request_id': 'req-d9f12b55-5ff6-4aeb-86eb-7d9f8417f9b6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:51.992094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d9f12b55-5ff6-4aeb-86eb-7d9f8417f9b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6d782400f1e432c8d10a31334c42e68', 'request_id': 'req-d9f12b55-5ff6-4aeb-86eb-7d9f8417f9b6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:52.005556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d9f12b55-5ff6-4aeb-86eb-7d9f8417f9b6 admin admin] The token cache is being invalidate because project e6d782400f1e432c8d10a31334c42e68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:23:52.005835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d9f12b55-5ff6-4aeb-86eb-7d9f8417f9b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:52.006439 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d9f12b55-5ff6-4aeb-86eb-7d9f8417f9b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:52.007901 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1317/2620] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:23:51 2022] DELETE /identity/v3/projects/e6d782400f1e432c8d10a31334c42e68 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:52.012784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffcdb0-bd78-4cb3-9fe0-5695d6f74208 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:52.013069 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffcdb0-bd78-4cb3-9fe0-5695d6f74208 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:52.013390 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ffcdb0-bd78-4cb3-9fe0-5695d6f74208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:52.033728 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a7ffcdb0-bd78-4cb3-9fe0-5695d6f74208 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:52.034924 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a7ffcdb0-bd78-4cb3-9fe0-5695d6f74208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:52.093032 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1304/2621] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:52.101835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802e095-ab68-47a2-a3a2-af3ccdf97de2 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:52.102113 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802e095-ab68-47a2-a3a2-af3ccdf97de2 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:52.102378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802e095-ab68-47a2-a3a2-af3ccdf97de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:52.110753 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:52.122892 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-7802e095-ab68-47a2-a3a2-af3ccdf97de2 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:52.124614 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7802e095-ab68-47a2-a3a2-af3ccdf97de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:52.132458 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1318/2622] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:23:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:52.136689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffcdb0-bd78-4cb3-9fe0-5695d6f74208 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:52.146928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ffcdb0-bd78-4cb3-9fe0-5695d6f74208 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:52.151498 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56fadb-539c-41ad-9a0a-7e24aa75a0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:52.152088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56fadb-539c-41ad-9a0a-7e24aa75a0fb admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:52.152365 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56fadb-539c-41ad-9a0a-7e24aa75a0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:52.152634 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b56fadb-539c-41ad-9a0a-7e24aa75a0fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:52.153312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56fadb-539c-41ad-9a0a-7e24aa75a0fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:23:52.153615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56fadb-539c-41ad-9a0a-7e24aa75a0fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:52.155219 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b56fadb-539c-41ad-9a0a-7e24aa75a0fb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:52.167375 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1305/2623] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:23:52.171521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802e095-ab68-47a2-a3a2-af3ccdf97de2 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:52.180756 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802e095-ab68-47a2-a3a2-af3ccdf97de2 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:52.185368 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffdf32-9780-40ad-b65d-9fbf4709a345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:52.185953 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffdf32-9780-40ad-b65d-9fbf4709a345 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:52.186226 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffdf32-9780-40ad-b65d-9fbf4709a345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:52.186490 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edffdf32-9780-40ad-b65d-9fbf4709a345 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:52.187109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffdf32-9780-40ad-b65d-9fbf4709a345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:52.189165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edffdf32-9780-40ad-b65d-9fbf4709a345 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:52.228900 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1319/2624] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:23:52 2022] POST /identity/v3/projects => generated 403 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:23:52.233098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56fadb-539c-41ad-9a0a-7e24aa75a0fb admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:52.243077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b56fadb-539c-41ad-9a0a-7e24aa75a0fb admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:52.272480 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:52.300600 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017a97c-070b-4194-9845-3d0991266572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:52.301220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017a97c-070b-4194-9845-3d0991266572 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:52.301506 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017a97c-070b-4194-9845-3d0991266572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:52.301775 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017a97c-070b-4194-9845-3d0991266572 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:52.302400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017a97c-070b-4194-9845-3d0991266572 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:52.304295 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017a97c-070b-4194-9845-3d0991266572 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:52.325598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1306/2625] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:23:52 2022] POST /identity/v3/users => generated 302 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:23:52.330505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffdf32-9780-40ad-b65d-9fbf4709a345 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:52.340926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edffdf32-9780-40ad-b65d-9fbf4709a345 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:52.345773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef801fb2-dfe7-4e80-9696-cdac414a8a56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:52.346358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef801fb2-dfe7-4e80-9696-cdac414a8a56 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:52.346635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef801fb2-dfe7-4e80-9696-cdac414a8a56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:52.346903 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef801fb2-dfe7-4e80-9696-cdac414a8a56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:52.347532 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef801fb2-dfe7-4e80-9696-cdac414a8a56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:52.349154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef801fb2-dfe7-4e80-9696-cdac414a8a56 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:52.358920 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1320/2626] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:23:52.362749 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017a97c-070b-4194-9845-3d0991266572 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:52.372543 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017a97c-070b-4194-9845-3d0991266572 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:52.377732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d8b76-15c6-41db-8584-73273761755a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:52.378318 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d8b76-15c6-41db-8584-73273761755a admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:52.378596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d8b76-15c6-41db-8584-73273761755a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:52.378869 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d8b76-15c6-41db-8584-73273761755a admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:52.379499 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d8b76-15c6-41db-8584-73273761755a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:52.381064 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d8b76-15c6-41db-8584-73273761755a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:52.390868 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1307/2627] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:23:52.395367 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef801fb2-dfe7-4e80-9696-cdac414a8a56 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:52.405073 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef801fb2-dfe7-4e80-9696-cdac414a8a56 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:52.410286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d22227-3640-469e-9479-6e6639101b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:52.410931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d22227-3640-469e-9479-6e6639101b0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:52.411212 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d22227-3640-469e-9479-6e6639101b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:52.411484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d22227-3640-469e-9479-6e6639101b0e admin admin] PATH_INFO: `/v3/projects/915d3a56ae5146eaa90eb2d6839ea499/users/fba5d9f34138437c9570a0f8ddf87dc7/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:52.425887 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d22227-3640-469e-9479-6e6639101b0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=915d3a56ae5146eaa90eb2d6839ea499, user_id=fba5d9f34138437c9570a0f8ddf87dc7, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:52.428947 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d22227-3640-469e-9479-6e6639101b0e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:52.442219 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1321/2628] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:23:52 2022] PUT /identity/v3/projects/915d3a56ae5146eaa90eb2d6839ea499/users/fba5d9f34138437c9570a0f8ddf87dc7/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:52.452478 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f535d-93da-40bf-b211-d273b04cedba None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:52.452769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f535d-93da-40bf-b211-d273b04cedba None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:52.453048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f535d-93da-40bf-b211-d273b04cedba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:52.482337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-3d1f535d-93da-40bf-b211-d273b04cedba None None] MFA Rules not processed for user `fba5d9f34138437c9570a0f8ddf87dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:52.483654 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3d1f535d-93da-40bf-b211-d273b04cedba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:52.548855 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1308/2629] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:23:52 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:52.553361 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d22227-3640-469e-9479-6e6639101b0e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:52.613523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d22227-3640-469e-9479-6e6639101b0e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:52.618173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d42054-773b-4c21-95cb-7f4b9deb977e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:52.618763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d42054-773b-4c21-95cb-7f4b9deb977e admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:52.619040 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d42054-773b-4c21-95cb-7f4b9deb977e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:52.619307 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d42054-773b-4c21-95cb-7f4b9deb977e admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:52.619932 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d42054-773b-4c21-95cb-7f4b9deb977e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:52.621835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d42054-773b-4c21-95cb-7f4b9deb977e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:52.647978 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1322/2630] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:23:52 2022] POST /identity/v3/projects => generated 405 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:23:52.651980 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f535d-93da-40bf-b211-d273b04cedba None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:52.662489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f535d-93da-40bf-b211-d273b04cedba None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:52.718859 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e9c38-1b32-4f3d-9653-26ad8e1e3d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:52.719514 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e9c38-1b32-4f3d-9653-26ad8e1e3d01 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:52.720525 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e9c38-1b32-4f3d-9653-26ad8e1e3d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:52.720791 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e9c38-1b32-4f3d-9653-26ad8e1e3d01 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:52.721433 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e9c38-1b32-4f3d-9653-26ad8e1e3d01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:52.723313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e9c38-1b32-4f3d-9653-26ad8e1e3d01 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:52.743657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1309/2631] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:23:52 2022] POST /identity/v3/users => generated 309 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:23:52.747601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d42054-773b-4c21-95cb-7f4b9deb977e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:52.757782 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d42054-773b-4c21-95cb-7f4b9deb977e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:52.762348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71511eb5-b52e-4cf6-84e1-575f579271d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:52.762933 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71511eb5-b52e-4cf6-84e1-575f579271d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:52.763210 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71511eb5-b52e-4cf6-84e1-575f579271d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:52.763479 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71511eb5-b52e-4cf6-84e1-575f579271d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:52.764105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71511eb5-b52e-4cf6-84e1-575f579271d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:52.765697 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71511eb5-b52e-4cf6-84e1-575f579271d6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:52.776444 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1323/2632] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:23:52.780326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e9c38-1b32-4f3d-9653-26ad8e1e3d01 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:52.790798 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e9c38-1b32-4f3d-9653-26ad8e1e3d01 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:52.795495 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aeb68d-14eb-458f-973b-e5124fa6a057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:52.796134 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aeb68d-14eb-458f-973b-e5124fa6a057 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:52.796884 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aeb68d-14eb-458f-973b-e5124fa6a057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:52.797209 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aeb68d-14eb-458f-973b-e5124fa6a057 admin admin] PATH_INFO: `/v3/projects/a38f618ca33b486fb2adf2bd38091b09/users/4877efd2a53e4b4d8836a41f4579d6cd/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:52.812456 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aeb68d-14eb-458f-973b-e5124fa6a057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a38f618ca33b486fb2adf2bd38091b09, user_id=4877efd2a53e4b4d8836a41f4579d6cd, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:52.815171 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aeb68d-14eb-458f-973b-e5124fa6a057 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:52.828258 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1310/2633] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:23:52 2022] PUT /identity/v3/projects/a38f618ca33b486fb2adf2bd38091b09/users/4877efd2a53e4b4d8836a41f4579d6cd/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:52.832667 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71511eb5-b52e-4cf6-84e1-575f579271d6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:52.842867 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71511eb5-b52e-4cf6-84e1-575f579271d6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:52.904647 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec90e3-17ca-4e2e-bfc6-af01c8163b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:52.905288 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec90e3-17ca-4e2e-bfc6-af01c8163b8c admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:52.905826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec90e3-17ca-4e2e-bfc6-af01c8163b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:52.905826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec90e3-17ca-4e2e-bfc6-af01c8163b8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:52.906450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec90e3-17ca-4e2e-bfc6-af01c8163b8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:52.908759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec90e3-17ca-4e2e-bfc6-af01c8163b8c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:52.918982 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1324/2634] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:23:52.923263 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aeb68d-14eb-458f-973b-e5124fa6a057 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:52.933857 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aeb68d-14eb-458f-973b-e5124fa6a057 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:52.938669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bbfcc-1cdc-4439-bb01-92107d81058f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:52.939317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bbfcc-1cdc-4439-bb01-92107d81058f admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:52.939596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bbfcc-1cdc-4439-bb01-92107d81058f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:52.939894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0bbfcc-1cdc-4439-bb01-92107d81058f admin admin] PATH_INFO: `/v3/projects/a38f618ca33b486fb2adf2bd38091b09/users/4877efd2a53e4b4d8836a41f4579d6cd/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:52.941833 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bbfcc-1cdc-4439-bb01-92107d81058f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a38f618ca33b486fb2adf2bd38091b09, user_id=4877efd2a53e4b4d8836a41f4579d6cd, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:52.944526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0bbfcc-1cdc-4439-bb01-92107d81058f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:52.946932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:52.958066 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1311/2635] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:23:52 2022] PUT /identity/v3/projects/a38f618ca33b486fb2adf2bd38091b09/users/4877efd2a53e4b4d8836a41f4579d6cd/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:52.970000 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156f551-1c82-45e9-9741-b25d38795a74 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:52.970566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156f551-1c82-45e9-9741-b25d38795a74 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:52.971373 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3156f551-1c82-45e9-9741-b25d38795a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:52.978382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:53.000552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-3156f551-1c82-45e9-9741-b25d38795a74 None None] MFA Rules not processed for user `4877efd2a53e4b4d8836a41f4579d6cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:53.001866 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3156f551-1c82-45e9-9741-b25d38795a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:53.070220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1325/2636] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:23:52 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:53.081448 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bbfcc-1cdc-4439-bb01-92107d81058f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:53.083169 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2a0bbfcc-1cdc-4439-bb01-92107d81058f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:53.160229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0bbfcc-1cdc-4439-bb01-92107d81058f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:53.163848 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21d8b3-6ad2-4112-919c-c7f9f972f303 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:53.164437 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21d8b3-6ad2-4112-919c-c7f9f972f303 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:53.164713 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21d8b3-6ad2-4112-919c-c7f9f972f303 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:53.164983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21d8b3-6ad2-4112-919c-c7f9f972f303 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:53.173849 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21d8b3-6ad2-4112-919c-c7f9f972f303 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:53.175295 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21d8b3-6ad2-4112-919c-c7f9f972f303 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:53.184638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1312/2637] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:53 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:53.341171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156f551-1c82-45e9-9741-b25d38795a74 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:53.401713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3156f551-1c82-45e9-9741-b25d38795a74 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:53.406245 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507b805-63a2-4292-bfe4-4cc6ed7b98d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:53.406883 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507b805-63a2-4292-bfe4-4cc6ed7b98d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:53.407159 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507b805-63a2-4292-bfe4-4cc6ed7b98d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:53.407427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f507b805-63a2-4292-bfe4-4cc6ed7b98d3 admin admin] PATH_INFO: `/v3/users/fba5d9f34138437c9570a0f8ddf87dc7` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:53.408519 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507b805-63a2-4292-bfe4-4cc6ed7b98d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fba5d9f34138437c9570a0f8ddf87dc7)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:53.410431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f507b805-63a2-4292-bfe4-4cc6ed7b98d3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:53.455737 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f507b805-63a2-4292-bfe4-4cc6ed7b98d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fba5d9f34138437c9570a0f8ddf87dc7'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:53.462574 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f507b805-63a2-4292-bfe4-4cc6ed7b98d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fba5d9f34138437c9570a0f8ddf87dc7'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:53.470752 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f507b805-63a2-4292-bfe4-4cc6ed7b98d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fba5d9f34138437c9570a0f8ddf87dc7'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:53.471425 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f507b805-63a2-4292-bfe4-4cc6ed7b98d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fba5d9f34138437c9570a0f8ddf87dc7'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:53.483092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-f507b805-63a2-4292-bfe4-4cc6ed7b98d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fba5d9f34138437c9570a0f8ddf87dc7'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:53.489509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1326/2638] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:23:53 2022] DELETE /identity/v3/users/fba5d9f34138437c9570a0f8ddf87dc7 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:53.493947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489386e1-7364-49b8-9dec-c4d6437ac9b0 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:53.494264 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489386e1-7364-49b8-9dec-c4d6437ac9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:53.494578 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489386e1-7364-49b8-9dec-c4d6437ac9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:53.513457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-489386e1-7364-49b8-9dec-c4d6437ac9b0 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:53.514572 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-489386e1-7364-49b8-9dec-c4d6437ac9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:53.570507 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1313/2639] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:23:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:53.580144 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507b805-63a2-4292-bfe4-4cc6ed7b98d3 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:53.581551 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f507b805-63a2-4292-bfe4-4cc6ed7b98d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:53.591420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:53.639846 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f507b805-63a2-4292-bfe4-4cc6ed7b98d3 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:53.643622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72811701-f7a2-4b5b-8eda-d7ec9c781e14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:53.644225 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811701-f7a2-4b5b-8eda-d7ec9c781e14 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:53.644503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811701-f7a2-4b5b-8eda-d7ec9c781e14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:53.644798 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72811701-f7a2-4b5b-8eda-d7ec9c781e14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:53.653618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72811701-f7a2-4b5b-8eda-d7ec9c781e14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:53.655035 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72811701-f7a2-4b5b-8eda-d7ec9c781e14 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:53.664207 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1327/2640] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:53.942820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489386e1-7364-49b8-9dec-c4d6437ac9b0 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:53.944279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-489386e1-7364-49b8-9dec-c4d6437ac9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:53.956483 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489386e1-7364-49b8-9dec-c4d6437ac9b0 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:53.961113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb04fa-6079-4c8d-886a-a24b34716b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:53.961774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb04fa-6079-4c8d-886a-a24b34716b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:53.962058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb04fa-6079-4c8d-886a-a24b34716b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:53.962347 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eb04fa-6079-4c8d-886a-a24b34716b18 admin admin] PATH_INFO: `/v3/projects/915d3a56ae5146eaa90eb2d6839ea499` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:53.963457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb04fa-6079-4c8d-886a-a24b34716b18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=915d3a56ae5146eaa90eb2d6839ea499)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:53.965388 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9eb04fa-6079-4c8d-886a-a24b34716b18 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:54.012349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d9eb04fa-6079-4c8d-886a-a24b34716b18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '915d3a56ae5146eaa90eb2d6839ea499', 'request_id': 'req-d9eb04fa-6079-4c8d-886a-a24b34716b18'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:54.019641 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d9eb04fa-6079-4c8d-886a-a24b34716b18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '915d3a56ae5146eaa90eb2d6839ea499', 'request_id': 'req-d9eb04fa-6079-4c8d-886a-a24b34716b18'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:54.025273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d9eb04fa-6079-4c8d-886a-a24b34716b18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '915d3a56ae5146eaa90eb2d6839ea499', 'request_id': 'req-d9eb04fa-6079-4c8d-886a-a24b34716b18'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:54.033640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:54.037685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d9eb04fa-6079-4c8d-886a-a24b34716b18 admin admin] The token cache is being invalidate because project 915d3a56ae5146eaa90eb2d6839ea499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:23:54.038126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d9eb04fa-6079-4c8d-886a-a24b34716b18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:54.039091 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-d9eb04fa-6079-4c8d-886a-a24b34716b18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:54.041221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1314/2641] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:23:53 2022] DELETE /identity/v3/projects/915d3a56ae5146eaa90eb2d6839ea499 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:54.045321 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72811701-f7a2-4b5b-8eda-d7ec9c781e14 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:54.055460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72811701-f7a2-4b5b-8eda-d7ec9c781e14 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:54.109234 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df1731-9f67-46f2-900a-ec4745d16a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:54.109869 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df1731-9f67-46f2-900a-ec4745d16a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:54.110779 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df1731-9f67-46f2-900a-ec4745d16a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:54.111050 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df1731-9f67-46f2-900a-ec4745d16a92 admin admin] PATH_INFO: `/v3/users/4877efd2a53e4b4d8836a41f4579d6cd` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:54.112151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df1731-9f67-46f2-900a-ec4745d16a92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4877efd2a53e4b4d8836a41f4579d6cd)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:54.114073 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df1731-9f67-46f2-900a-ec4745d16a92 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:54.159279 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-31df1731-9f67-46f2-900a-ec4745d16a92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4877efd2a53e4b4d8836a41f4579d6cd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:54.165990 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-31df1731-9f67-46f2-900a-ec4745d16a92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4877efd2a53e4b4d8836a41f4579d6cd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:54.174069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-31df1731-9f67-46f2-900a-ec4745d16a92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4877efd2a53e4b4d8836a41f4579d6cd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:54.174614 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-31df1731-9f67-46f2-900a-ec4745d16a92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4877efd2a53e4b4d8836a41f4579d6cd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:54.186713 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-31df1731-9f67-46f2-900a-ec4745d16a92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4877efd2a53e4b4d8836a41f4579d6cd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:54.192492 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1328/2642] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:23:54 2022] DELETE /identity/v3/users/4877efd2a53e4b4d8836a41f4579d6cd => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:54.460638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb04fa-6079-4c8d-886a-a24b34716b18 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:54.462282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d9eb04fa-6079-4c8d-886a-a24b34716b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:54.479245 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eb04fa-6079-4c8d-886a-a24b34716b18 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:54.538176 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb3986-bd54-4999-ac0d-9005e3b6368d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:54.538733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb3986-bd54-4999-ac0d-9005e3b6368d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:54.539012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb3986-bd54-4999-ac0d-9005e3b6368d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:54.539284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb3986-bd54-4999-ac0d-9005e3b6368d admin admin] PATH_INFO: `/v3/projects/a38f618ca33b486fb2adf2bd38091b09` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:54.540391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb3986-bd54-4999-ac0d-9005e3b6368d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a38f618ca33b486fb2adf2bd38091b09)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:54.542325 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb3986-bd54-4999-ac0d-9005e3b6368d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:54.588393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f9cb3986-bd54-4999-ac0d-9005e3b6368d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a38f618ca33b486fb2adf2bd38091b09', 'request_id': 'req-f9cb3986-bd54-4999-ac0d-9005e3b6368d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:54.595820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f9cb3986-bd54-4999-ac0d-9005e3b6368d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a38f618ca33b486fb2adf2bd38091b09', 'request_id': 'req-f9cb3986-bd54-4999-ac0d-9005e3b6368d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:54.601428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f9cb3986-bd54-4999-ac0d-9005e3b6368d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a38f618ca33b486fb2adf2bd38091b09', 'request_id': 'req-f9cb3986-bd54-4999-ac0d-9005e3b6368d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:54.612041 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f9cb3986-bd54-4999-ac0d-9005e3b6368d admin admin] The token cache is being invalidate because project a38f618ca33b486fb2adf2bd38091b09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:23:54.612322 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f9cb3986-bd54-4999-ac0d-9005e3b6368d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:54.612928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f9cb3986-bd54-4999-ac0d-9005e3b6368d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:54.614425 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1315/2643] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:23:54 2022] DELETE /identity/v3/projects/a38f618ca33b486fb2adf2bd38091b09 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:54.619478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93484c9-ca8a-4045-ac06-c047a62454cf None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:54.619763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93484c9-ca8a-4045-ac06-c047a62454cf None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:54.620037 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93484c9-ca8a-4045-ac06-c047a62454cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:54.639464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c93484c9-ca8a-4045-ac06-c047a62454cf None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:54.640860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c93484c9-ca8a-4045-ac06-c047a62454cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:54.664505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:54.701627 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1329/2644] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:54.710070 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7b430-cfd9-425f-ab6a-7f609183bcb7 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:54.710351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7b430-cfd9-425f-ab6a-7f609183bcb7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:54.710624 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7b430-cfd9-425f-ab6a-7f609183bcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:54.730520 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a7e7b430-cfd9-425f-ab6a-7f609183bcb7 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:54.731702 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a7e7b430-cfd9-425f-ab6a-7f609183bcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:54.739470 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1316/2645] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:54.743564 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93484c9-ca8a-4045-ac06-c047a62454cf None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:54.753761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93484c9-ca8a-4045-ac06-c047a62454cf None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:54.758375 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8327a84-eda9-4a31-99f6-bebd294bb8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:54.758967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8327a84-eda9-4a31-99f6-bebd294bb8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:54.759245 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8327a84-eda9-4a31-99f6-bebd294bb8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:54.759517 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8327a84-eda9-4a31-99f6-bebd294bb8b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:54.760150 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8327a84-eda9-4a31-99f6-bebd294bb8b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:23:54.760449 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8327a84-eda9-4a31-99f6-bebd294bb8b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:54.762056 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8327a84-eda9-4a31-99f6-bebd294bb8b9 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:54.774033 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1330/2646] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:23:54.778101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7b430-cfd9-425f-ab6a-7f609183bcb7 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:54.788129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7b430-cfd9-425f-ab6a-7f609183bcb7 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:54.793148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee288a1-7e1a-493d-be83-ae999aceb5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:54.793716 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee288a1-7e1a-493d-be83-ae999aceb5e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:54.793991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee288a1-7e1a-493d-be83-ae999aceb5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:54.794260 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee288a1-7e1a-493d-be83-ae999aceb5e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:54.794874 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee288a1-7e1a-493d-be83-ae999aceb5e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:54.796755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee288a1-7e1a-493d-be83-ae999aceb5e6 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:54.824951 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1317/2647] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:23:54 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:23:54.829695 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8327a84-eda9-4a31-99f6-bebd294bb8b9 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:54.840099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8327a84-eda9-4a31-99f6-bebd294bb8b9 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:54.895730 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3aa4b5-4408-43bf-a28b-d1494259c8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:54.896325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3aa4b5-4408-43bf-a28b-d1494259c8e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:54.896604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3aa4b5-4408-43bf-a28b-d1494259c8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:54.896879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3aa4b5-4408-43bf-a28b-d1494259c8e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:54.897536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3aa4b5-4408-43bf-a28b-d1494259c8e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:54.899433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3aa4b5-4408-43bf-a28b-d1494259c8e2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:54.920782 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1331/2648] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:23:54 2022] POST /identity/v3/users => generated 281 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:23:54.925438 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee288a1-7e1a-493d-be83-ae999aceb5e6 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:54.935521 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee288a1-7e1a-493d-be83-ae999aceb5e6 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:54.940821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e98c88-d549-4b89-903c-5d25be1a2058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:54.941436 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e98c88-d549-4b89-903c-5d25be1a2058 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:54.941712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e98c88-d549-4b89-903c-5d25be1a2058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:54.941979 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e98c88-d549-4b89-903c-5d25be1a2058 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:54.942605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e98c88-d549-4b89-903c-5d25be1a2058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:54.944174 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e98c88-d549-4b89-903c-5d25be1a2058 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:54.954349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1318/2649] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:23:54.958114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3aa4b5-4408-43bf-a28b-d1494259c8e2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:54.968034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3aa4b5-4408-43bf-a28b-d1494259c8e2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:54.972991 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3dd18d-8c65-49b8-95e0-9e1b1b2120e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:54.973625 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3dd18d-8c65-49b8-95e0-9e1b1b2120e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:54.973897 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3dd18d-8c65-49b8-95e0-9e1b1b2120e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:54.974167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3dd18d-8c65-49b8-95e0-9e1b1b2120e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:54.974805 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3dd18d-8c65-49b8-95e0-9e1b1b2120e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:54.976371 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3dd18d-8c65-49b8-95e0-9e1b1b2120e1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:54.987595 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1332/2650] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:23:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:23:54.991530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e98c88-d549-4b89-903c-5d25be1a2058 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:55.002119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e98c88-d549-4b89-903c-5d25be1a2058 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:55.006808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd8021-2d58-45a5-9293-c3bd3b4329d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:55.007461 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd8021-2d58-45a5-9293-c3bd3b4329d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:55.007715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd8021-2d58-45a5-9293-c3bd3b4329d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:55.007981 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cd8021-2d58-45a5-9293-c3bd3b4329d7 admin admin] PATH_INFO: `/v3/projects/a62f0a04282847b49e96c1fb7a2b5d94/users/f5b179514f44421e826a723da291be4a/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:55.022242 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd8021-2d58-45a5-9293-c3bd3b4329d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a62f0a04282847b49e96c1fb7a2b5d94, user_id=f5b179514f44421e826a723da291be4a, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:55.025637 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cd8021-2d58-45a5-9293-c3bd3b4329d7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:55.039263 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1319/2651] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:23:54 2022] PUT /identity/v3/projects/a62f0a04282847b49e96c1fb7a2b5d94/users/f5b179514f44421e826a723da291be4a/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:55.051299 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674d56c-7119-4c0f-a4f0-2eb9a680b787 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:55.052038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674d56c-7119-4c0f-a4f0-2eb9a680b787 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:55.052240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674d56c-7119-4c0f-a4f0-2eb9a680b787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:55.079574 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4674d56c-7119-4c0f-a4f0-2eb9a680b787 None None] MFA Rules not processed for user `f5b179514f44421e826a723da291be4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:55.080761 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4674d56c-7119-4c0f-a4f0-2eb9a680b787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:55.142938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1333/2652] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:23:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:55.147381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd8021-2d58-45a5-9293-c3bd3b4329d7 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:55.177060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:55.210699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cd8021-2d58-45a5-9293-c3bd3b4329d7 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:55.215401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4d9a3-6f94-4ad3-8564-81f35c42a823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:55.215999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4d9a3-6f94-4ad3-8564-81f35c42a823 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:55.216272 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4d9a3-6f94-4ad3-8564-81f35c42a823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:55.216538 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4d9a3-6f94-4ad3-8564-81f35c42a823 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:55.217211 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4d9a3-6f94-4ad3-8564-81f35c42a823 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:55.219156 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4d9a3-6f94-4ad3-8564-81f35c42a823 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:55.245938 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1320/2653] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:23:55 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:23:55.249937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674d56c-7119-4c0f-a4f0-2eb9a680b787 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:55.261270 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4674d56c-7119-4c0f-a4f0-2eb9a680b787 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:55.280690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:55.320834 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072e524-17b2-4d2e-a914-b68e82d3b055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:55.321478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072e524-17b2-4d2e-a914-b68e82d3b055 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:55.321757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072e524-17b2-4d2e-a914-b68e82d3b055 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:55.322027 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072e524-17b2-4d2e-a914-b68e82d3b055 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:55.322661 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072e524-17b2-4d2e-a914-b68e82d3b055 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:55.324598 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072e524-17b2-4d2e-a914-b68e82d3b055 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:55.344996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1334/2654] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:23:55 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:23:55.349175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4d9a3-6f94-4ad3-8564-81f35c42a823 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:55.359301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4d9a3-6f94-4ad3-8564-81f35c42a823 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:55.364665 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83253b7e-7804-4f19-b518-6404fc7186d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:55.365281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83253b7e-7804-4f19-b518-6404fc7186d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:55.365556 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83253b7e-7804-4f19-b518-6404fc7186d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:55.365820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83253b7e-7804-4f19-b518-6404fc7186d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:55.366472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83253b7e-7804-4f19-b518-6404fc7186d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:55.368033 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83253b7e-7804-4f19-b518-6404fc7186d7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:55.378486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1321/2655] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:23:55.382588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072e524-17b2-4d2e-a914-b68e82d3b055 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:55.392424 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072e524-17b2-4d2e-a914-b68e82d3b055 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:55.397280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9eb33a-fe07-4fff-9abf-c252c30e2bde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:55.397921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9eb33a-fe07-4fff-9abf-c252c30e2bde admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:55.398200 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9eb33a-fe07-4fff-9abf-c252c30e2bde admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:55.398474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9eb33a-fe07-4fff-9abf-c252c30e2bde admin admin] PATH_INFO: `/v3/projects/17be514a84c6410ca6b41bb25c7f02f7/users/b53901ceab9c4d2089f42fea38da0dbd/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:55.413495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9eb33a-fe07-4fff-9abf-c252c30e2bde admin admin] RBAC: Authorizing `identity:create_grant(project_id=17be514a84c6410ca6b41bb25c7f02f7, user_id=b53901ceab9c4d2089f42fea38da0dbd, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:55.416112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9eb33a-fe07-4fff-9abf-c252c30e2bde admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:55.429053 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1335/2656] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:23:55 2022] PUT /identity/v3/projects/17be514a84c6410ca6b41bb25c7f02f7/users/b53901ceab9c4d2089f42fea38da0dbd/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:55.432921 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83253b7e-7804-4f19-b518-6404fc7186d7 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:55.443154 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83253b7e-7804-4f19-b518-6404fc7186d7 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:55.497745 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9100020-d52d-4d87-9a0a-d5ca081ee184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:55.498358 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9100020-d52d-4d87-9a0a-d5ca081ee184 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:55.498629 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9100020-d52d-4d87-9a0a-d5ca081ee184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:55.498895 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9100020-d52d-4d87-9a0a-d5ca081ee184 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:55.499511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9100020-d52d-4d87-9a0a-d5ca081ee184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:55.501097 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9100020-d52d-4d87-9a0a-d5ca081ee184 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:55.510440 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1322/2657] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:23:55 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:23:55.514680 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9eb33a-fe07-4fff-9abf-c252c30e2bde admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:55.524880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9eb33a-fe07-4fff-9abf-c252c30e2bde admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:55.529416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9f786-5eae-47fd-a340-4cd4f8cb72a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:55.530051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9f786-5eae-47fd-a340-4cd4f8cb72a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:55.530324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9f786-5eae-47fd-a340-4cd4f8cb72a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:55.530591 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9f786-5eae-47fd-a340-4cd4f8cb72a7 admin admin] PATH_INFO: `/v3/projects/17be514a84c6410ca6b41bb25c7f02f7/users/b53901ceab9c4d2089f42fea38da0dbd/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:55.533008 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9f786-5eae-47fd-a340-4cd4f8cb72a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17be514a84c6410ca6b41bb25c7f02f7, user_id=b53901ceab9c4d2089f42fea38da0dbd, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:55.535633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9f786-5eae-47fd-a340-4cd4f8cb72a7 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:55.545475 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1336/2658] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:23:55 2022] PUT /identity/v3/projects/17be514a84c6410ca6b41bb25c7f02f7/users/b53901ceab9c4d2089f42fea38da0dbd/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:55.556012 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26399739-fac8-48ca-b18c-ed260a09ee2e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:55.556234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26399739-fac8-48ca-b18c-ed260a09ee2e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:55.556511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26399739-fac8-48ca-b18c-ed260a09ee2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:55.583611 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-26399739-fac8-48ca-b18c-ed260a09ee2e None None] MFA Rules not processed for user `b53901ceab9c4d2089f42fea38da0dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:55.584797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-26399739-fac8-48ca-b18c-ed260a09ee2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:55.652529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1323/2659] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:23:55 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:55.668367 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9f786-5eae-47fd-a340-4cd4f8cb72a7 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:55.670233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-37c9f786-5eae-47fd-a340-4cd4f8cb72a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:55.727967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9f786-5eae-47fd-a340-4cd4f8cb72a7 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:55.731597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c648ff3b-537e-4a24-871a-d09fa49e57d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:55.732190 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648ff3b-537e-4a24-871a-d09fa49e57d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:55.732468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648ff3b-537e-4a24-871a-d09fa49e57d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:55.732735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c648ff3b-537e-4a24-871a-d09fa49e57d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:55.741547 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648ff3b-537e-4a24-871a-d09fa49e57d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:55.742984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c648ff3b-537e-4a24-871a-d09fa49e57d4 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:55.752282 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1337/2660] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:56.163167 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26399739-fac8-48ca-b18c-ed260a09ee2e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:56.202941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:56.241165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26399739-fac8-48ca-b18c-ed260a09ee2e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:56.245699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cf5f6-45ba-4770-a4d5-f8ad194d16ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:56.246325 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cf5f6-45ba-4770-a4d5-f8ad194d16ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:56.246602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cf5f6-45ba-4770-a4d5-f8ad194d16ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:56.246871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1cf5f6-45ba-4770-a4d5-f8ad194d16ac admin admin] PATH_INFO: `/v3/users/f5b179514f44421e826a723da291be4a` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:56.247989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cf5f6-45ba-4770-a4d5-f8ad194d16ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5b179514f44421e826a723da291be4a)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:56.249936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1cf5f6-45ba-4770-a4d5-f8ad194d16ac admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:56.297865 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8c1cf5f6-45ba-4770-a4d5-f8ad194d16ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5b179514f44421e826a723da291be4a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:56.304671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8c1cf5f6-45ba-4770-a4d5-f8ad194d16ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5b179514f44421e826a723da291be4a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:56.316697 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8c1cf5f6-45ba-4770-a4d5-f8ad194d16ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5b179514f44421e826a723da291be4a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:56.317389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8c1cf5f6-45ba-4770-a4d5-f8ad194d16ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5b179514f44421e826a723da291be4a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:56.317992 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8c1cf5f6-45ba-4770-a4d5-f8ad194d16ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5b179514f44421e826a723da291be4a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:56.330998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1324/2661] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:23:56 2022] DELETE /identity/v3/users/f5b179514f44421e826a723da291be4a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:56.335509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811b469-8ae5-49e5-b3bd-c3b1301c611c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:56.336069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811b469-8ae5-49e5-b3bd-c3b1301c611c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:56.336069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f811b469-8ae5-49e5-b3bd-c3b1301c611c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:56.354861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f811b469-8ae5-49e5-b3bd-c3b1301c611c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:56.356041 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f811b469-8ae5-49e5-b3bd-c3b1301c611c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:56.413170 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1338/2662] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:23:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:56.423416 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cf5f6-45ba-4770-a4d5-f8ad194d16ac admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:56.424793 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8c1cf5f6-45ba-4770-a4d5-f8ad194d16ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:56.479925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1cf5f6-45ba-4770-a4d5-f8ad194d16ac admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:56.483722 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed3ca8-57fb-499d-8cd3-87375ab6e032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:56.484326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3ca8-57fb-499d-8cd3-87375ab6e032 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:56.484604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3ca8-57fb-499d-8cd3-87375ab6e032 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:56.485424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebed3ca8-57fb-499d-8cd3-87375ab6e032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:56.495626 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed3ca8-57fb-499d-8cd3-87375ab6e032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:56.497174 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebed3ca8-57fb-499d-8cd3-87375ab6e032 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:56.507503 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1325/2663] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:56.771582 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811b469-8ae5-49e5-b3bd-c3b1301c611c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:56.773098 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f811b469-8ae5-49e5-b3bd-c3b1301c611c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:56.787639 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f811b469-8ae5-49e5-b3bd-c3b1301c611c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:56.793836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939baea4-c4a7-4c8f-bdb9-75c750be4364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:56.794546 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939baea4-c4a7-4c8f-bdb9-75c750be4364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:56.794720 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939baea4-c4a7-4c8f-bdb9-75c750be4364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:56.795025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939baea4-c4a7-4c8f-bdb9-75c750be4364 admin admin] PATH_INFO: `/v3/projects/a62f0a04282847b49e96c1fb7a2b5d94` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:56.796073 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939baea4-c4a7-4c8f-bdb9-75c750be4364 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a62f0a04282847b49e96c1fb7a2b5d94)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:56.798461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939baea4-c4a7-4c8f-bdb9-75c750be4364 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:56.815593 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:56.847519 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-939baea4-c4a7-4c8f-bdb9-75c750be4364 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a62f0a04282847b49e96c1fb7a2b5d94', 'request_id': 'req-939baea4-c4a7-4c8f-bdb9-75c750be4364'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:56.855943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-939baea4-c4a7-4c8f-bdb9-75c750be4364 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a62f0a04282847b49e96c1fb7a2b5d94', 'request_id': 'req-939baea4-c4a7-4c8f-bdb9-75c750be4364'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:56.862219 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-939baea4-c4a7-4c8f-bdb9-75c750be4364 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a62f0a04282847b49e96c1fb7a2b5d94', 'request_id': 'req-939baea4-c4a7-4c8f-bdb9-75c750be4364'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:56.874036 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-939baea4-c4a7-4c8f-bdb9-75c750be4364 admin admin] The token cache is being invalidate because project a62f0a04282847b49e96c1fb7a2b5d94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:23:56.874322 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-939baea4-c4a7-4c8f-bdb9-75c750be4364 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:56.874939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-939baea4-c4a7-4c8f-bdb9-75c750be4364 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:56.876456 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1339/2664] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:23:56 2022] DELETE /identity/v3/projects/a62f0a04282847b49e96c1fb7a2b5d94 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:56.880589 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed3ca8-57fb-499d-8cd3-87375ab6e032 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:56.890215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebed3ca8-57fb-499d-8cd3-87375ab6e032 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:56.931945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:56.947854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea90e49-b568-479e-b319-d82e65b41690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:56.948477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea90e49-b568-479e-b319-d82e65b41690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:56.948759 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea90e49-b568-479e-b319-d82e65b41690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:56.949030 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea90e49-b568-479e-b319-d82e65b41690 admin admin] PATH_INFO: `/v3/users/b53901ceab9c4d2089f42fea38da0dbd` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:56.950193 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea90e49-b568-479e-b319-d82e65b41690 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b53901ceab9c4d2089f42fea38da0dbd)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:56.952180 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea90e49-b568-479e-b319-d82e65b41690 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:56.998712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5ea90e49-b568-479e-b319-d82e65b41690 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b53901ceab9c4d2089f42fea38da0dbd'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:57.005627 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5ea90e49-b568-479e-b319-d82e65b41690 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b53901ceab9c4d2089f42fea38da0dbd'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:57.017257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5ea90e49-b568-479e-b319-d82e65b41690 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b53901ceab9c4d2089f42fea38da0dbd'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:57.017917 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5ea90e49-b568-479e-b319-d82e65b41690 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b53901ceab9c4d2089f42fea38da0dbd'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:57.018515 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5ea90e49-b568-479e-b319-d82e65b41690 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b53901ceab9c4d2089f42fea38da0dbd'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:57.030934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1326/2665] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:23:56 2022] DELETE /identity/v3/users/b53901ceab9c4d2089f42fea38da0dbd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:57.301046 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939baea4-c4a7-4c8f-bdb9-75c750be4364 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:57.302516 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-939baea4-c4a7-4c8f-bdb9-75c750be4364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:57.317653 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939baea4-c4a7-4c8f-bdb9-75c750be4364 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:57.374493 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b51fa5-5634-4340-a3cb-bc6415ee52a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:57.375147 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b51fa5-5634-4340-a3cb-bc6415ee52a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:57.375384 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b51fa5-5634-4340-a3cb-bc6415ee52a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:57.375636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b51fa5-5634-4340-a3cb-bc6415ee52a4 admin admin] PATH_INFO: `/v3/projects/17be514a84c6410ca6b41bb25c7f02f7` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:57.376800 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b51fa5-5634-4340-a3cb-bc6415ee52a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17be514a84c6410ca6b41bb25c7f02f7)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:57.378733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b51fa5-5634-4340-a3cb-bc6415ee52a4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:57.424151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-01b51fa5-5634-4340-a3cb-bc6415ee52a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17be514a84c6410ca6b41bb25c7f02f7', 'request_id': 'req-01b51fa5-5634-4340-a3cb-bc6415ee52a4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:57.430135 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-01b51fa5-5634-4340-a3cb-bc6415ee52a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17be514a84c6410ca6b41bb25c7f02f7', 'request_id': 'req-01b51fa5-5634-4340-a3cb-bc6415ee52a4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:57.435648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-01b51fa5-5634-4340-a3cb-bc6415ee52a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17be514a84c6410ca6b41bb25c7f02f7', 'request_id': 'req-01b51fa5-5634-4340-a3cb-bc6415ee52a4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:57.448225 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-01b51fa5-5634-4340-a3cb-bc6415ee52a4 admin admin] The token cache is being invalidate because project 17be514a84c6410ca6b41bb25c7f02f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:23:57.448493 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-01b51fa5-5634-4340-a3cb-bc6415ee52a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:57.449106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-01b51fa5-5634-4340-a3cb-bc6415ee52a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:23:57.450580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1340/2666] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:23:57 2022] DELETE /identity/v3/projects/17be514a84c6410ca6b41bb25c7f02f7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:57.456726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc18ff7-047f-40c6-8380-a6124c2966a5 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:57.457339 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc18ff7-047f-40c6-8380-a6124c2966a5 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:57.457339 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc18ff7-047f-40c6-8380-a6124c2966a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:57.478682 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9bc18ff7-047f-40c6-8380-a6124c2966a5 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:57.479888 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9bc18ff7-047f-40c6-8380-a6124c2966a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:57.537519 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1327/2667] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:57.545372 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcadcd2-cabf-4d15-8d29-06dbdd3c102c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:57.545667 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcadcd2-cabf-4d15-8d29-06dbdd3c102c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:57.545943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcadcd2-cabf-4d15-8d29-06dbdd3c102c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:57.565623 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-cbcadcd2-cabf-4d15-8d29-06dbdd3c102c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:57.566810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cbcadcd2-cabf-4d15-8d29-06dbdd3c102c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:57.574861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1341/2668] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:57.579002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc18ff7-047f-40c6-8380-a6124c2966a5 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:57.589212 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc18ff7-047f-40c6-8380-a6124c2966a5 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:57.593759 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d2369-7808-4d97-b100-d675c148f44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:57.594348 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d2369-7808-4d97-b100-d675c148f44e admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:57.594624 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d2369-7808-4d97-b100-d675c148f44e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:57.594897 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d2369-7808-4d97-b100-d675c148f44e admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:57.595869 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d2369-7808-4d97-b100-d675c148f44e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:23:57.596166 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d2369-7808-4d97-b100-d675c148f44e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:57.597777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d2369-7808-4d97-b100-d675c148f44e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:57.609077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1328/2669] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:23:57.613235 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcadcd2-cabf-4d15-8d29-06dbdd3c102c None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:57.622543 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcadcd2-cabf-4d15-8d29-06dbdd3c102c None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:57.627656 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492405ef-8e92-4476-af23-5a9e4b2f8a6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:57.628204 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492405ef-8e92-4476-af23-5a9e4b2f8a6f admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:57.628480 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492405ef-8e92-4476-af23-5a9e4b2f8a6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:57.628748 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492405ef-8e92-4476-af23-5a9e4b2f8a6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:57.629383 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492405ef-8e92-4476-af23-5a9e4b2f8a6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:57.631247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492405ef-8e92-4476-af23-5a9e4b2f8a6f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:57.658335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1342/2670] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:23:57 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:23:57.662376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d2369-7808-4d97-b100-d675c148f44e admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:57.672547 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d2369-7808-4d97-b100-d675c148f44e admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:57.718800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:57.730379 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefcb90-bb4d-4f13-9da3-574ac3c8b917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:57.730967 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefcb90-bb4d-4f13-9da3-574ac3c8b917 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:57.731237 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefcb90-bb4d-4f13-9da3-574ac3c8b917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:57.731499 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcefcb90-bb4d-4f13-9da3-574ac3c8b917 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:57.732125 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefcb90-bb4d-4f13-9da3-574ac3c8b917 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:57.734024 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcefcb90-bb4d-4f13-9da3-574ac3c8b917 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:57.754616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1329/2671] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:23:57 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:23:57.758737 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492405ef-8e92-4476-af23-5a9e4b2f8a6f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:57.765600 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:57.769860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492405ef-8e92-4476-af23-5a9e4b2f8a6f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:57.775820 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b00aed-f94a-4ad2-b8b9-6d6978074a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:57.776568 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b00aed-f94a-4ad2-b8b9-6d6978074a1b admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:57.777003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b00aed-f94a-4ad2-b8b9-6d6978074a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:57.777366 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b00aed-f94a-4ad2-b8b9-6d6978074a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:57.778213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b00aed-f94a-4ad2-b8b9-6d6978074a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:57.779830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b00aed-f94a-4ad2-b8b9-6d6978074a1b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:57.789651 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1343/2672] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:23:57.793673 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefcb90-bb4d-4f13-9da3-574ac3c8b917 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:57.803608 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcefcb90-bb4d-4f13-9da3-574ac3c8b917 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:57.808156 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ac6d48-52dd-4815-8a24-f43b288ed9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:57.808768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac6d48-52dd-4815-8a24-f43b288ed9cd admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:57.809044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac6d48-52dd-4815-8a24-f43b288ed9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:57.809351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ac6d48-52dd-4815-8a24-f43b288ed9cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:57.809990 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac6d48-52dd-4815-8a24-f43b288ed9cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:57.811558 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ac6d48-52dd-4815-8a24-f43b288ed9cd admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:57.822471 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1330/2673] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:23:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:23:57.826372 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b00aed-f94a-4ad2-b8b9-6d6978074a1b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:57.836972 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b00aed-f94a-4ad2-b8b9-6d6978074a1b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:57.841531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9c38c-c5df-4e2c-b5a1-207ef110181b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:57.842170 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9c38c-c5df-4e2c-b5a1-207ef110181b admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:57.842446 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9c38c-c5df-4e2c-b5a1-207ef110181b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:57.842722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9c38c-c5df-4e2c-b5a1-207ef110181b admin admin] PATH_INFO: `/v3/projects/4e994fa287fd44e0b483d74edfaba91c/users/5c9426efd12046a0b5b04c8285bb7b12/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:57.857359 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9c38c-c5df-4e2c-b5a1-207ef110181b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e994fa287fd44e0b483d74edfaba91c, user_id=5c9426efd12046a0b5b04c8285bb7b12, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:57.860004 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9c38c-c5df-4e2c-b5a1-207ef110181b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:57.872575 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1344/2674] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:23:57 2022] PUT /identity/v3/projects/4e994fa287fd44e0b483d74edfaba91c/users/5c9426efd12046a0b5b04c8285bb7b12/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:57.884479 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ea0ab-4318-4ea3-b2e6-9a5e88ef0674 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:57.884702 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ea0ab-4318-4ea3-b2e6-9a5e88ef0674 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:57.884975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ea0ab-4318-4ea3-b2e6-9a5e88ef0674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:57.912347 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-370ea0ab-4318-4ea3-b2e6-9a5e88ef0674 None None] MFA Rules not processed for user `5c9426efd12046a0b5b04c8285bb7b12`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:57.913575 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-370ea0ab-4318-4ea3-b2e6-9a5e88ef0674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:57.976144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1331/2675] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:23:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:57.980619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9c38c-c5df-4e2c-b5a1-207ef110181b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:58.041531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9c38c-c5df-4e2c-b5a1-207ef110181b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:58.046020 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f59cd7-3bd7-4ed8-af42-2de0d09af4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:58.046612 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f59cd7-3bd7-4ed8-af42-2de0d09af4cd admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:58.046911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f59cd7-3bd7-4ed8-af42-2de0d09af4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:58.047178 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f59cd7-3bd7-4ed8-af42-2de0d09af4cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:58.047793 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f59cd7-3bd7-4ed8-af42-2de0d09af4cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:58.050194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f59cd7-3bd7-4ed8-af42-2de0d09af4cd admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:58.076362 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1345/2676] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:23:57 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:23:58.080387 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ea0ab-4318-4ea3-b2e6-9a5e88ef0674 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:58.090910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ea0ab-4318-4ea3-b2e6-9a5e88ef0674 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:58.146711 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7408886-59e0-4199-b04e-32ceee8d3e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:58.147236 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7408886-59e0-4199-b04e-32ceee8d3e40 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:58.147508 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7408886-59e0-4199-b04e-32ceee8d3e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:58.147776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7408886-59e0-4199-b04e-32ceee8d3e40 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:58.148406 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7408886-59e0-4199-b04e-32ceee8d3e40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:58.150419 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7408886-59e0-4199-b04e-32ceee8d3e40 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:58.170922 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1332/2677] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:23:58 2022] POST /identity/v3/users => generated 297 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:23:58.175919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f59cd7-3bd7-4ed8-af42-2de0d09af4cd admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:58.186069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f59cd7-3bd7-4ed8-af42-2de0d09af4cd admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:58.191313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41761de-2317-4cd7-aa83-4c5b11c1c393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:58.191905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41761de-2317-4cd7-aa83-4c5b11c1c393 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:58.192182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41761de-2317-4cd7-aa83-4c5b11c1c393 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:58.192448 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41761de-2317-4cd7-aa83-4c5b11c1c393 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:58.193081 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41761de-2317-4cd7-aa83-4c5b11c1c393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:58.194698 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41761de-2317-4cd7-aa83-4c5b11c1c393 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:58.205369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1346/2678] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:23:58.209444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7408886-59e0-4199-b04e-32ceee8d3e40 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:58.219252 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7408886-59e0-4199-b04e-32ceee8d3e40 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:58.224031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9bc8f-8e6b-4791-bfb8-af3067296d71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:58.224686 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9bc8f-8e6b-4791-bfb8-af3067296d71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:58.224943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9bc8f-8e6b-4791-bfb8-af3067296d71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:58.225242 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c9bc8f-8e6b-4791-bfb8-af3067296d71 admin admin] PATH_INFO: `/v3/projects/d0c886816ea946808c847bd9df7ed533/users/f1ababf02410468c851c7885407833d9/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:58.240486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9bc8f-8e6b-4791-bfb8-af3067296d71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c886816ea946808c847bd9df7ed533, user_id=f1ababf02410468c851c7885407833d9, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:58.243238 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c9bc8f-8e6b-4791-bfb8-af3067296d71 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:58.256634 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1333/2679] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:23:58 2022] PUT /identity/v3/projects/d0c886816ea946808c847bd9df7ed533/users/f1ababf02410468c851c7885407833d9/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:58.260740 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41761de-2317-4cd7-aa83-4c5b11c1c393 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:58.271223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41761de-2317-4cd7-aa83-4c5b11c1c393 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:58.326801 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdc7d18-53a7-48b2-83f9-3427e49e5f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:58.327395 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc7d18-53a7-48b2-83f9-3427e49e5f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:58.327674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc7d18-53a7-48b2-83f9-3427e49e5f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:58.327941 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdc7d18-53a7-48b2-83f9-3427e49e5f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:58.328564 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc7d18-53a7-48b2-83f9-3427e49e5f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:58.330839 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdc7d18-53a7-48b2-83f9-3427e49e5f67 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:58.340472 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1347/2680] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:23:58.344659 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9bc8f-8e6b-4791-bfb8-af3067296d71 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:58.355221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c9bc8f-8e6b-4791-bfb8-af3067296d71 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:58.359810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2baac-17cf-43d5-b10a-47564233c928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:58.360451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2baac-17cf-43d5-b10a-47564233c928 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:58.360728 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2baac-17cf-43d5-b10a-47564233c928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:58.361019 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b2baac-17cf-43d5-b10a-47564233c928 admin admin] PATH_INFO: `/v3/projects/d0c886816ea946808c847bd9df7ed533/users/f1ababf02410468c851c7885407833d9/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:58.362932 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2baac-17cf-43d5-b10a-47564233c928 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c886816ea946808c847bd9df7ed533, user_id=f1ababf02410468c851c7885407833d9, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:58.365583 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b2baac-17cf-43d5-b10a-47564233c928 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:58.375552 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1334/2681] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:23:58 2022] PUT /identity/v3/projects/d0c886816ea946808c847bd9df7ed533/users/f1ababf02410468c851c7885407833d9/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:23:58.386147 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f11578-780f-4749-b845-b5bff35d02ca None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:58.386635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f11578-780f-4749-b845-b5bff35d02ca None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:58.386746 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f11578-780f-4749-b845-b5bff35d02ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:58.414009 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-a1f11578-780f-4749-b845-b5bff35d02ca None None] MFA Rules not processed for user `f1ababf02410468c851c7885407833d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:23:58.415141 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-a1f11578-780f-4749-b845-b5bff35d02ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:58.443679 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:58.483335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1348/2682] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:23:58 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:23:58.497733 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2baac-17cf-43d5-b10a-47564233c928 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:58.498184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-d7b2baac-17cf-43d5-b10a-47564233c928 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:23:58.516335 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:23:58.560406 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b2baac-17cf-43d5-b10a-47564233c928 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:58.563989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7871489-0534-42fc-a5c1-073515cceebf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:58.565132 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7871489-0534-42fc-a5c1-073515cceebf service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:58.565461 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7871489-0534-42fc-a5c1-073515cceebf service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:58.565729 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7871489-0534-42fc-a5c1-073515cceebf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:58.574462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7871489-0534-42fc-a5c1-073515cceebf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:58.576217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7871489-0534-42fc-a5c1-073515cceebf service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:58.586368 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1335/2683] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:58 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:23:58.686352 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f11578-780f-4749-b845-b5bff35d02ca None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:23:58.696431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f11578-780f-4749-b845-b5bff35d02ca None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:23:58.699925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4050d67b-1a05-4eed-8881-327438252e31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:23:58.700505 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050d67b-1a05-4eed-8881-327438252e31 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:23:58.700779 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050d67b-1a05-4eed-8881-327438252e31 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:23:58.701043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4050d67b-1a05-4eed-8881-327438252e31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:23:58.756700 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050d67b-1a05-4eed-8881-327438252e31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:23:58.758152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4050d67b-1a05-4eed-8881-327438252e31 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:23:58.766749 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1349/2684] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:23:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:01.343835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7871489-0534-42fc-a5c1-073515cceebf service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:01.404502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7871489-0534-42fc-a5c1-073515cceebf service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:01.409005 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ee14c-634a-4130-9b7f-90fd2a231cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:01.409649 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ee14c-634a-4130-9b7f-90fd2a231cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:01.409920 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ee14c-634a-4130-9b7f-90fd2a231cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:01.410182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ee14c-634a-4130-9b7f-90fd2a231cd0 admin admin] PATH_INFO: `/v3/users/5c9426efd12046a0b5b04c8285bb7b12` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:01.411293 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ee14c-634a-4130-9b7f-90fd2a231cd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c9426efd12046a0b5b04c8285bb7b12)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:01.413218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ee14c-634a-4130-9b7f-90fd2a231cd0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:01.456757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e03ee14c-634a-4130-9b7f-90fd2a231cd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c9426efd12046a0b5b04c8285bb7b12'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:01.463552 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e03ee14c-634a-4130-9b7f-90fd2a231cd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c9426efd12046a0b5b04c8285bb7b12'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:01.475670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e03ee14c-634a-4130-9b7f-90fd2a231cd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c9426efd12046a0b5b04c8285bb7b12'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:01.476329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e03ee14c-634a-4130-9b7f-90fd2a231cd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c9426efd12046a0b5b04c8285bb7b12'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:01.476910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e03ee14c-634a-4130-9b7f-90fd2a231cd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c9426efd12046a0b5b04c8285bb7b12'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:01.491098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1336/2685] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:24:01 2022] DELETE /identity/v3/users/5c9426efd12046a0b5b04c8285bb7b12 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:01.495742 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8487f6-af4f-4351-b97d-2989e7987905 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:01.496250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8487f6-af4f-4351-b97d-2989e7987905 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:01.496364 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8487f6-af4f-4351-b97d-2989e7987905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:01.515637 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-6b8487f6-af4f-4351-b97d-2989e7987905 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:01.516830 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6b8487f6-af4f-4351-b97d-2989e7987905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:01.578075 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1350/2686] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:01.588597 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ee14c-634a-4130-9b7f-90fd2a231cd0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:01.590047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e03ee14c-634a-4130-9b7f-90fd2a231cd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:01.626075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:01.649866 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ee14c-634a-4130-9b7f-90fd2a231cd0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:01.653513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00a2c9-3ba8-4cf4-8590-df9664ca6e6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:01.654110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00a2c9-3ba8-4cf4-8590-df9664ca6e6b service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:01.654389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00a2c9-3ba8-4cf4-8590-df9664ca6e6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:01.654654 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff00a2c9-3ba8-4cf4-8590-df9664ca6e6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:01.663773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00a2c9-3ba8-4cf4-8590-df9664ca6e6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:01.665204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff00a2c9-3ba8-4cf4-8590-df9664ca6e6b service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:01.676274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1337/2687] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:01.940934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8487f6-af4f-4351-b97d-2989e7987905 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:01.942399 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6b8487f6-af4f-4351-b97d-2989e7987905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:01.954486 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8487f6-af4f-4351-b97d-2989e7987905 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:01.959203 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caeaa99-9865-4d22-862e-612e1bffadde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:01.959832 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caeaa99-9865-4d22-862e-612e1bffadde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:01.960112 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caeaa99-9865-4d22-862e-612e1bffadde admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:01.960403 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1caeaa99-9865-4d22-862e-612e1bffadde admin admin] PATH_INFO: `/v3/projects/4e994fa287fd44e0b483d74edfaba91c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:01.961527 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caeaa99-9865-4d22-862e-612e1bffadde admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e994fa287fd44e0b483d74edfaba91c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:01.963461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1caeaa99-9865-4d22-862e-612e1bffadde admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:02.010241 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1caeaa99-9865-4d22-862e-612e1bffadde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e994fa287fd44e0b483d74edfaba91c', 'request_id': 'req-1caeaa99-9865-4d22-862e-612e1bffadde'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:02.016301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1caeaa99-9865-4d22-862e-612e1bffadde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e994fa287fd44e0b483d74edfaba91c', 'request_id': 'req-1caeaa99-9865-4d22-862e-612e1bffadde'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:02.020293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:02.022821 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1caeaa99-9865-4d22-862e-612e1bffadde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e994fa287fd44e0b483d74edfaba91c', 'request_id': 'req-1caeaa99-9865-4d22-862e-612e1bffadde'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:02.035818 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1caeaa99-9865-4d22-862e-612e1bffadde admin admin] The token cache is being invalidate because project 4e994fa287fd44e0b483d74edfaba91c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:24:02.036103 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1caeaa99-9865-4d22-862e-612e1bffadde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:02.036727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1caeaa99-9865-4d22-862e-612e1bffadde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:02.038237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1351/2688] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:24:01 2022] DELETE /identity/v3/projects/4e994fa287fd44e0b483d74edfaba91c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:24:02.042360 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00a2c9-3ba8-4cf4-8590-df9664ca6e6b service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:02.053181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff00a2c9-3ba8-4cf4-8590-df9664ca6e6b service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:02.109170 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e11de33-6ab6-453c-9604-62500ec7905c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:02.109817 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11de33-6ab6-453c-9604-62500ec7905c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:02.110095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11de33-6ab6-453c-9604-62500ec7905c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:02.110366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e11de33-6ab6-453c-9604-62500ec7905c admin admin] PATH_INFO: `/v3/users/f1ababf02410468c851c7885407833d9` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:02.112087 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11de33-6ab6-453c-9604-62500ec7905c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1ababf02410468c851c7885407833d9)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:02.114047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e11de33-6ab6-453c-9604-62500ec7905c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:02.158748 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5e11de33-6ab6-453c-9604-62500ec7905c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1ababf02410468c851c7885407833d9'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:02.165508 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5e11de33-6ab6-453c-9604-62500ec7905c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1ababf02410468c851c7885407833d9'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:02.177920 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5e11de33-6ab6-453c-9604-62500ec7905c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1ababf02410468c851c7885407833d9'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:02.178601 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5e11de33-6ab6-453c-9604-62500ec7905c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1ababf02410468c851c7885407833d9'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:02.179192 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5e11de33-6ab6-453c-9604-62500ec7905c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1ababf02410468c851c7885407833d9'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:02.191561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1338/2689] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:24:02 2022] DELETE /identity/v3/users/f1ababf02410468c851c7885407833d9 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:02.462038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caeaa99-9865-4d22-862e-612e1bffadde admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:02.463563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1caeaa99-9865-4d22-862e-612e1bffadde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:02.479233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1caeaa99-9865-4d22-862e-612e1bffadde admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:02.535601 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9746a-6617-4156-8860-4d0cde9e4fe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:02.536322 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9746a-6617-4156-8860-4d0cde9e4fe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:02.536461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9746a-6617-4156-8860-4d0cde9e4fe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:02.536719 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9746a-6617-4156-8860-4d0cde9e4fe3 admin admin] PATH_INFO: `/v3/projects/d0c886816ea946808c847bd9df7ed533` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:02.537890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9746a-6617-4156-8860-4d0cde9e4fe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0c886816ea946808c847bd9df7ed533)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:02.539989 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9746a-6617-4156-8860-4d0cde9e4fe3 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:02.584647 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-71e9746a-6617-4156-8860-4d0cde9e4fe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0c886816ea946808c847bd9df7ed533', 'request_id': 'req-71e9746a-6617-4156-8860-4d0cde9e4fe3'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:02.590621 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-71e9746a-6617-4156-8860-4d0cde9e4fe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0c886816ea946808c847bd9df7ed533', 'request_id': 'req-71e9746a-6617-4156-8860-4d0cde9e4fe3'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:02.596890 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-71e9746a-6617-4156-8860-4d0cde9e4fe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0c886816ea946808c847bd9df7ed533', 'request_id': 'req-71e9746a-6617-4156-8860-4d0cde9e4fe3'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:02.608523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-71e9746a-6617-4156-8860-4d0cde9e4fe3 admin admin] The token cache is being invalidate because project d0c886816ea946808c847bd9df7ed533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:24:02.608844 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-71e9746a-6617-4156-8860-4d0cde9e4fe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:02.610122 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-71e9746a-6617-4156-8860-4d0cde9e4fe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:02.611628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1352/2690] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:24:02 2022] DELETE /identity/v3/projects/d0c886816ea946808c847bd9df7ed533 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:02.616776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f320086-8e51-4cc6-8de6-ad30a6634b16 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:02.617292 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f320086-8e51-4cc6-8de6-ad30a6634b16 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:02.617489 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f320086-8e51-4cc6-8de6-ad30a6634b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:02.636451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-7f320086-8e51-4cc6-8de6-ad30a6634b16 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:02.637693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7f320086-8e51-4cc6-8de6-ad30a6634b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:02.686594 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:02.699878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1339/2691] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 31 19:24:02.707939 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456587e0-e070-47c0-b610-45b395f2cab1 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:02.708224 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456587e0-e070-47c0-b610-45b395f2cab1 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:02.708500 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456587e0-e070-47c0-b610-45b395f2cab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:02.728530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-456587e0-e070-47c0-b610-45b395f2cab1 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:02.729769 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-456587e0-e070-47c0-b610-45b395f2cab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:02.737639 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1353/2692] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:02.741749 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f320086-8e51-4cc6-8de6-ad30a6634b16 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:02.751422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f320086-8e51-4cc6-8de6-ad30a6634b16 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:02.756026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2306e-4739-4c33-8437-0a21c1f548af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:02.756616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2306e-4739-4c33-8437-0a21c1f548af admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:02.756895 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2306e-4739-4c33-8437-0a21c1f548af admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:02.757166 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff2306e-4739-4c33-8437-0a21c1f548af admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:02.757858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2306e-4739-4c33-8437-0a21c1f548af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:24:02.758163 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2306e-4739-4c33-8437-0a21c1f548af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:02.759730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff2306e-4739-4c33-8437-0a21c1f548af admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:02.771551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1340/2693] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:24:02.775744 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456587e0-e070-47c0-b610-45b395f2cab1 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:02.785520 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456587e0-e070-47c0-b610-45b395f2cab1 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:02.790126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb41059-a8df-4a4e-a387-720b17e832ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:02.790660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb41059-a8df-4a4e-a387-720b17e832ff admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:02.790938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb41059-a8df-4a4e-a387-720b17e832ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:02.791205 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb41059-a8df-4a4e-a387-720b17e832ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:02.791817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb41059-a8df-4a4e-a387-720b17e832ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:02.793712 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb41059-a8df-4a4e-a387-720b17e832ff admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:02.821331 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1354/2694] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:02 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:02.825235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2306e-4739-4c33-8437-0a21c1f548af admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:02.835633 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff2306e-4739-4c33-8437-0a21c1f548af admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:02.890190 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15450f-3eb9-46ce-a1d9-9129250a45f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:02.890779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15450f-3eb9-46ce-a1d9-9129250a45f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:02.891049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15450f-3eb9-46ce-a1d9-9129250a45f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:02.891312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa15450f-3eb9-46ce-a1d9-9129250a45f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:02.891926 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15450f-3eb9-46ce-a1d9-9129250a45f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:02.894464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa15450f-3eb9-46ce-a1d9-9129250a45f0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:02.914701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1341/2695] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:02 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:02.918833 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb41059-a8df-4a4e-a387-720b17e832ff admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:02.928959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb41059-a8df-4a4e-a387-720b17e832ff admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:02.933537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6df77-5e60-4325-9723-861e103e0c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:02.934120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6df77-5e60-4325-9723-861e103e0c98 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:02.934394 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6df77-5e60-4325-9723-861e103e0c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:02.934659 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6df77-5e60-4325-9723-861e103e0c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:02.935286 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6df77-5e60-4325-9723-861e103e0c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:02.936835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c6df77-5e60-4325-9723-861e103e0c98 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:02.947253 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1355/2696] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:02.950974 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15450f-3eb9-46ce-a1d9-9129250a45f0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:02.961553 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa15450f-3eb9-46ce-a1d9-9129250a45f0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:02.966112 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4658aded-a381-4563-ac07-464f223b4a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:02.966696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658aded-a381-4563-ac07-464f223b4a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:02.966967 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658aded-a381-4563-ac07-464f223b4a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:02.967233 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4658aded-a381-4563-ac07-464f223b4a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:02.967855 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4658aded-a381-4563-ac07-464f223b4a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:02.969442 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4658aded-a381-4563-ac07-464f223b4a78 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:02.979915 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1342/2697] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:02.983925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6df77-5e60-4325-9723-861e103e0c98 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:02.993942 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6df77-5e60-4325-9723-861e103e0c98 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:02.998605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fbed3-f48c-4aeb-87c7-2fb30d0705dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:02.999272 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fbed3-f48c-4aeb-87c7-2fb30d0705dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:02.999549 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fbed3-f48c-4aeb-87c7-2fb30d0705dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:02.999815 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73fbed3-f48c-4aeb-87c7-2fb30d0705dc admin admin] PATH_INFO: `/v3/projects/8d0a1b2528c64b7c9af6dd17931c52ff/users/be421d1639c04dccafbc6bb4711fd9e6/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:03.014489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73fbed3-f48c-4aeb-87c7-2fb30d0705dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d0a1b2528c64b7c9af6dd17931c52ff, user_id=be421d1639c04dccafbc6bb4711fd9e6, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:03.017079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73fbed3-f48c-4aeb-87c7-2fb30d0705dc admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:03.030103 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1356/2698] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:02 2022] PUT /identity/v3/projects/8d0a1b2528c64b7c9af6dd17931c52ff/users/be421d1639c04dccafbc6bb4711fd9e6/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:03.041411 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dbe5a-d678-491e-8a92-5dd0ed9e136e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:03.041706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dbe5a-d678-491e-8a92-5dd0ed9e136e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:03.042045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016dbe5a-d678-491e-8a92-5dd0ed9e136e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:03.070060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-016dbe5a-d678-491e-8a92-5dd0ed9e136e None None] MFA Rules not processed for user `be421d1639c04dccafbc6bb4711fd9e6`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:03.071268 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-016dbe5a-d678-491e-8a92-5dd0ed9e136e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:03.134178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1343/2699] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:03 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:03.138697 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fbed3-f48c-4aeb-87c7-2fb30d0705dc admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:03.152309 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:03.202055 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73fbed3-f48c-4aeb-87c7-2fb30d0705dc admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:03.206690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db7bdc-e248-47b5-b5c4-ff2d6cd63348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:03.207290 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db7bdc-e248-47b5-b5c4-ff2d6cd63348 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:03.207570 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db7bdc-e248-47b5-b5c4-ff2d6cd63348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:03.207841 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57db7bdc-e248-47b5-b5c4-ff2d6cd63348 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:03.208464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db7bdc-e248-47b5-b5c4-ff2d6cd63348 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:03.210466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57db7bdc-e248-47b5-b5c4-ff2d6cd63348 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:03.237593 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1357/2700] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:03 2022] POST /identity/v3/projects => generated 367 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:03.241822 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dbe5a-d678-491e-8a92-5dd0ed9e136e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:03.253517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016dbe5a-d678-491e-8a92-5dd0ed9e136e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:03.299926 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:03.310595 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d3f96-6685-49b5-b37d-15a3f3826b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:03.311221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d3f96-6685-49b5-b37d-15a3f3826b88 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:03.311494 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d3f96-6685-49b5-b37d-15a3f3826b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:03.311872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d3f96-6685-49b5-b37d-15a3f3826b88 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:03.312554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d3f96-6685-49b5-b37d-15a3f3826b88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:03.315181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763d3f96-6685-49b5-b37d-15a3f3826b88 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:03.335247 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1344/2701] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:03 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:03.339309 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db7bdc-e248-47b5-b5c4-ff2d6cd63348 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:03.350013 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57db7bdc-e248-47b5-b5c4-ff2d6cd63348 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:03.354616 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f47a6-4e20-44af-8bf3-daa7a88c5412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:03.355223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f47a6-4e20-44af-8bf3-daa7a88c5412 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:03.355487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f47a6-4e20-44af-8bf3-daa7a88c5412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:03.355756 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908f47a6-4e20-44af-8bf3-daa7a88c5412 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:03.356388 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f47a6-4e20-44af-8bf3-daa7a88c5412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:03.358003 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908f47a6-4e20-44af-8bf3-daa7a88c5412 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:03.368891 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1358/2702] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:03.372973 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d3f96-6685-49b5-b37d-15a3f3826b88 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:03.383764 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d3f96-6685-49b5-b37d-15a3f3826b88 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:03.388385 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f43502-b7a6-475e-a9e5-0e9296824758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:03.389022 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f43502-b7a6-475e-a9e5-0e9296824758 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:03.389337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f43502-b7a6-475e-a9e5-0e9296824758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:03.389611 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f43502-b7a6-475e-a9e5-0e9296824758 admin admin] PATH_INFO: `/v3/projects/0b19157d51a94f6dbd318850d7252694/users/9fa071802b9c47fb923a984a3add8a82/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:03.404164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f43502-b7a6-475e-a9e5-0e9296824758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b19157d51a94f6dbd318850d7252694, user_id=9fa071802b9c47fb923a984a3add8a82, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:03.406855 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f43502-b7a6-475e-a9e5-0e9296824758 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:03.420496 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1345/2703] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:03 2022] PUT /identity/v3/projects/0b19157d51a94f6dbd318850d7252694/users/9fa071802b9c47fb923a984a3add8a82/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:24:03.424884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f47a6-4e20-44af-8bf3-daa7a88c5412 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:03.434750 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908f47a6-4e20-44af-8bf3-daa7a88c5412 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:03.490051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6608fa4-f9ce-4181-aee4-c3d437895cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:03.490639 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6608fa4-f9ce-4181-aee4-c3d437895cbc admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:03.490911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6608fa4-f9ce-4181-aee4-c3d437895cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:03.491175 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6608fa4-f9ce-4181-aee4-c3d437895cbc admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:03.491802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6608fa4-f9ce-4181-aee4-c3d437895cbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:03.493454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6608fa4-f9ce-4181-aee4-c3d437895cbc admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:03.502900 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1359/2704] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:03.506988 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f43502-b7a6-475e-a9e5-0e9296824758 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:03.517173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f43502-b7a6-475e-a9e5-0e9296824758 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:03.522254 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd2c75-3ccd-4e56-a14d-8e885dbff46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:03.522894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd2c75-3ccd-4e56-a14d-8e885dbff46a admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:03.523169 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd2c75-3ccd-4e56-a14d-8e885dbff46a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:03.523438 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd2c75-3ccd-4e56-a14d-8e885dbff46a admin admin] PATH_INFO: `/v3/projects/0b19157d51a94f6dbd318850d7252694/users/9fa071802b9c47fb923a984a3add8a82/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:03.525368 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd2c75-3ccd-4e56-a14d-8e885dbff46a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b19157d51a94f6dbd318850d7252694, user_id=9fa071802b9c47fb923a984a3add8a82, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:03.528011 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd2c75-3ccd-4e56-a14d-8e885dbff46a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:03.538695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1346/2705] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:03 2022] PUT /identity/v3/projects/0b19157d51a94f6dbd318850d7252694/users/9fa071802b9c47fb923a984a3add8a82/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:03.548596 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff604e-5d56-4259-9d9f-3a056782821e None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:03.549089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff604e-5d56-4259-9d9f-3a056782821e None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:03.549214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edff604e-5d56-4259-9d9f-3a056782821e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:03.577559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-edff604e-5d56-4259-9d9f-3a056782821e None None] MFA Rules not processed for user `9fa071802b9c47fb923a984a3add8a82`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:03.578714 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-edff604e-5d56-4259-9d9f-3a056782821e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:03.646261 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1360/2706] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:03 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:03.657383 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd2c75-3ccd-4e56-a14d-8e885dbff46a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:03.659193 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-79cd2c75-3ccd-4e56-a14d-8e885dbff46a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:03.718699 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd2c75-3ccd-4e56-a14d-8e885dbff46a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:03.722831 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabce0f-5bb6-4238-a24d-a156523530de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:03.723428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabce0f-5bb6-4238-a24d-a156523530de service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:03.723705 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabce0f-5bb6-4238-a24d-a156523530de service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:03.723975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cabce0f-5bb6-4238-a24d-a156523530de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:03.732789 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabce0f-5bb6-4238-a24d-a156523530de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:03.734264 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cabce0f-5bb6-4238-a24d-a156523530de service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:03.743544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1347/2707] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:03 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:03.822929 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff604e-5d56-4259-9d9f-3a056782821e None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:03.833105 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edff604e-5d56-4259-9d9f-3a056782821e None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:03.839987 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697506d8-076b-45a5-a533-1ba093c95086 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:03.840647 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697506d8-076b-45a5-a533-1ba093c95086 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:03.840922 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697506d8-076b-45a5-a533-1ba093c95086 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:03.841208 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697506d8-076b-45a5-a533-1ba093c95086 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:03.846868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:03.900076 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697506d8-076b-45a5-a533-1ba093c95086 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:03.901556 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697506d8-076b-45a5-a533-1ba093c95086 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:03.910530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1361/2708] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:08.571630 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabce0f-5bb6-4238-a24d-a156523530de service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:08.764256 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cabce0f-5bb6-4238-a24d-a156523530de service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:08.768706 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e27a0-32a4-4b79-95e6-f92236445e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:08.769403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e27a0-32a4-4b79-95e6-f92236445e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:08.769675 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e27a0-32a4-4b79-95e6-f92236445e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:08.769947 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904e27a0-32a4-4b79-95e6-f92236445e3d admin admin] PATH_INFO: `/v3/users/be421d1639c04dccafbc6bb4711fd9e6` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:08.771042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e27a0-32a4-4b79-95e6-f92236445e3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=be421d1639c04dccafbc6bb4711fd9e6)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:08.772936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904e27a0-32a4-4b79-95e6-f92236445e3d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:08.817231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-904e27a0-32a4-4b79-95e6-f92236445e3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be421d1639c04dccafbc6bb4711fd9e6'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:08.818444 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:08.826241 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-904e27a0-32a4-4b79-95e6-f92236445e3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be421d1639c04dccafbc6bb4711fd9e6'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:08.838532 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-904e27a0-32a4-4b79-95e6-f92236445e3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be421d1639c04dccafbc6bb4711fd9e6'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:08.839226 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-904e27a0-32a4-4b79-95e6-f92236445e3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be421d1639c04dccafbc6bb4711fd9e6'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:08.839830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-904e27a0-32a4-4b79-95e6-f92236445e3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be421d1639c04dccafbc6bb4711fd9e6'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:08.854941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1348/2709] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:24:08 2022] DELETE /identity/v3/users/be421d1639c04dccafbc6bb4711fd9e6 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:08.859603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ea763-a87e-4b2e-b75e-236863b01c93 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:08.860086 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ea763-a87e-4b2e-b75e-236863b01c93 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:08.860223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9ea763-a87e-4b2e-b75e-236863b01c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:08.881431 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-af9ea763-a87e-4b2e-b75e-236863b01c93 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:08.882631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-af9ea763-a87e-4b2e-b75e-236863b01c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:08.940037 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1362/2710] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:08.949759 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e27a0-32a4-4b79-95e6-f92236445e3d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:08.951140 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-904e27a0-32a4-4b79-95e6-f92236445e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:09.008014 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904e27a0-32a4-4b79-95e6-f92236445e3d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:09.011734 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6257e-f016-4542-a60f-dde98dd7cb29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:09.012329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6257e-f016-4542-a60f-dde98dd7cb29 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:09.012603 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6257e-f016-4542-a60f-dde98dd7cb29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:09.012870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a6257e-f016-4542-a60f-dde98dd7cb29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:09.021702 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6257e-f016-4542-a60f-dde98dd7cb29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:09.023045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a6257e-f016-4542-a60f-dde98dd7cb29 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:09.032551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1349/2711] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:09.300071 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ea763-a87e-4b2e-b75e-236863b01c93 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:09.301563 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-af9ea763-a87e-4b2e-b75e-236863b01c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:09.314728 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9ea763-a87e-4b2e-b75e-236863b01c93 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:09.319717 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84997774-123f-4453-a030-d1eeb69137d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:09.320425 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84997774-123f-4453-a030-d1eeb69137d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:09.320779 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84997774-123f-4453-a030-d1eeb69137d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:09.321346 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84997774-123f-4453-a030-d1eeb69137d4 admin admin] PATH_INFO: `/v3/projects/8d0a1b2528c64b7c9af6dd17931c52ff` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:09.322536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84997774-123f-4453-a030-d1eeb69137d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d0a1b2528c64b7c9af6dd17931c52ff)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:09.324635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84997774-123f-4453-a030-d1eeb69137d4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:09.378576 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-84997774-123f-4453-a030-d1eeb69137d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d0a1b2528c64b7c9af6dd17931c52ff', 'request_id': 'req-84997774-123f-4453-a030-d1eeb69137d4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:09.384842 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-84997774-123f-4453-a030-d1eeb69137d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d0a1b2528c64b7c9af6dd17931c52ff', 'request_id': 'req-84997774-123f-4453-a030-d1eeb69137d4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:09.390415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-84997774-123f-4453-a030-d1eeb69137d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d0a1b2528c64b7c9af6dd17931c52ff', 'request_id': 'req-84997774-123f-4453-a030-d1eeb69137d4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:09.403760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-84997774-123f-4453-a030-d1eeb69137d4 admin admin] The token cache is being invalidate because project 8d0a1b2528c64b7c9af6dd17931c52ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:24:09.404051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-84997774-123f-4453-a030-d1eeb69137d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:09.404785 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-84997774-123f-4453-a030-d1eeb69137d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:09.406536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1363/2712] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:24:09 2022] DELETE /identity/v3/projects/8d0a1b2528c64b7c9af6dd17931c52ff => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:09.411083 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6257e-f016-4542-a60f-dde98dd7cb29 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:09.421357 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a6257e-f016-4542-a60f-dde98dd7cb29 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:09.482830 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a417e9-9988-481c-a22e-23a500698189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:09.483777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a417e9-9988-481c-a22e-23a500698189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:09.484139 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a417e9-9988-481c-a22e-23a500698189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:09.484469 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a417e9-9988-481c-a22e-23a500698189 admin admin] PATH_INFO: `/v3/users/9fa071802b9c47fb923a984a3add8a82` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:09.485605 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a417e9-9988-481c-a22e-23a500698189 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fa071802b9c47fb923a984a3add8a82)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:09.487511 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a417e9-9988-481c-a22e-23a500698189 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:09.533056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-72a417e9-9988-481c-a22e-23a500698189 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fa071802b9c47fb923a984a3add8a82'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:09.540172 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-72a417e9-9988-481c-a22e-23a500698189 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fa071802b9c47fb923a984a3add8a82'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:09.550346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:09.552906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-72a417e9-9988-481c-a22e-23a500698189 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fa071802b9c47fb923a984a3add8a82'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:09.553975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-72a417e9-9988-481c-a22e-23a500698189 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fa071802b9c47fb923a984a3add8a82'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:09.554708 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-72a417e9-9988-481c-a22e-23a500698189 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fa071802b9c47fb923a984a3add8a82'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:09.568330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1350/2713] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:24:09 2022] DELETE /identity/v3/users/9fa071802b9c47fb923a984a3add8a82 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:09.836308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84997774-123f-4453-a030-d1eeb69137d4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:09.837835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-84997774-123f-4453-a030-d1eeb69137d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:09.851223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84997774-123f-4453-a030-d1eeb69137d4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:09.882932 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:09.912395 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f307f-9da2-44cb-8d73-cd43675a4c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:09.913018 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f307f-9da2-44cb-8d73-cd43675a4c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:09.913324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f307f-9da2-44cb-8d73-cd43675a4c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:09.913596 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f307f-9da2-44cb-8d73-cd43675a4c54 admin admin] PATH_INFO: `/v3/projects/0b19157d51a94f6dbd318850d7252694` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:09.914693 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f307f-9da2-44cb-8d73-cd43675a4c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b19157d51a94f6dbd318850d7252694)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:09.916574 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59f307f-9da2-44cb-8d73-cd43675a4c54 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:09.964146 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d59f307f-9da2-44cb-8d73-cd43675a4c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b19157d51a94f6dbd318850d7252694', 'request_id': 'req-d59f307f-9da2-44cb-8d73-cd43675a4c54'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:09.970422 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d59f307f-9da2-44cb-8d73-cd43675a4c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b19157d51a94f6dbd318850d7252694', 'request_id': 'req-d59f307f-9da2-44cb-8d73-cd43675a4c54'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:09.975949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d59f307f-9da2-44cb-8d73-cd43675a4c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b19157d51a94f6dbd318850d7252694', 'request_id': 'req-d59f307f-9da2-44cb-8d73-cd43675a4c54'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:09.988617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d59f307f-9da2-44cb-8d73-cd43675a4c54 admin admin] The token cache is being invalidate because project 0b19157d51a94f6dbd318850d7252694 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:24:09.988903 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d59f307f-9da2-44cb-8d73-cd43675a4c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:09.989531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-d59f307f-9da2-44cb-8d73-cd43675a4c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:09.990977 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1364/2714] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:24:09 2022] DELETE /identity/v3/projects/0b19157d51a94f6dbd318850d7252694 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:09.996153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d502e5-7961-4d35-8ba8-8be382d4232e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:09.996778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d502e5-7961-4d35-8ba8-8be382d4232e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:09.996778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d502e5-7961-4d35-8ba8-8be382d4232e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:10.016241 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-54d502e5-7961-4d35-8ba8-8be382d4232e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:10.017490 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-54d502e5-7961-4d35-8ba8-8be382d4232e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:10.074283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1351/2715] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:10.083258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cf938-d8d0-4360-ba29-66b91430085f None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:10.083487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cf938-d8d0-4360-ba29-66b91430085f None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:10.083766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6cf938-d8d0-4360-ba29-66b91430085f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:10.103529 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-4c6cf938-d8d0-4360-ba29-66b91430085f None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:10.104721 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-4c6cf938-d8d0-4360-ba29-66b91430085f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:10.113094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1365/2716] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:10.117175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d502e5-7961-4d35-8ba8-8be382d4232e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:10.126681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d502e5-7961-4d35-8ba8-8be382d4232e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:10.131863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44a554-b092-411b-b9bd-0e840b8c4971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:10.132454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44a554-b092-411b-b9bd-0e840b8c4971 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:10.132732 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44a554-b092-411b-b9bd-0e840b8c4971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:10.133004 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c44a554-b092-411b-b9bd-0e840b8c4971 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:10.133681 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44a554-b092-411b-b9bd-0e840b8c4971 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:24:10.133980 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44a554-b092-411b-b9bd-0e840b8c4971 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:10.135545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c44a554-b092-411b-b9bd-0e840b8c4971 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:10.146913 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1352/2717] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:24:10.151070 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cf938-d8d0-4360-ba29-66b91430085f None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:10.160233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6cf938-d8d0-4360-ba29-66b91430085f None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:10.164772 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d286ca8-d199-43d8-bd51-690f763a7933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:10.165380 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d286ca8-d199-43d8-bd51-690f763a7933 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:10.165903 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d286ca8-d199-43d8-bd51-690f763a7933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:10.166157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d286ca8-d199-43d8-bd51-690f763a7933 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:10.166999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d286ca8-d199-43d8-bd51-690f763a7933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:10.168876 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d286ca8-d199-43d8-bd51-690f763a7933 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:10.197096 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1366/2718] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:10 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:10.200805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44a554-b092-411b-b9bd-0e840b8c4971 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:10.210855 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c44a554-b092-411b-b9bd-0e840b8c4971 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:10.266499 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a08ac5-7aa5-4fdc-8cc2-3afdcf824913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:10.267174 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a08ac5-7aa5-4fdc-8cc2-3afdcf824913 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:10.267481 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a08ac5-7aa5-4fdc-8cc2-3afdcf824913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:10.267756 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a08ac5-7aa5-4fdc-8cc2-3afdcf824913 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:10.268394 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a08ac5-7aa5-4fdc-8cc2-3afdcf824913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:10.270525 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a08ac5-7aa5-4fdc-8cc2-3afdcf824913 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:10.291683 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1353/2719] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:10 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:24:10.295879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d286ca8-d199-43d8-bd51-690f763a7933 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:10.306686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d286ca8-d199-43d8-bd51-690f763a7933 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:10.313293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1c970-337c-484d-95e3-40b3665d5691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:10.313903 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1c970-337c-484d-95e3-40b3665d5691 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:10.314179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1c970-337c-484d-95e3-40b3665d5691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:10.314444 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc1c970-337c-484d-95e3-40b3665d5691 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:10.315077 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc1c970-337c-484d-95e3-40b3665d5691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:10.316704 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc1c970-337c-484d-95e3-40b3665d5691 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:10.326849 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1367/2720] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:10.330936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a08ac5-7aa5-4fdc-8cc2-3afdcf824913 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:10.340945 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a08ac5-7aa5-4fdc-8cc2-3afdcf824913 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:10.345810 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf290c-2f6e-4097-a2a6-7d49019f07cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:10.346395 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf290c-2f6e-4097-a2a6-7d49019f07cf admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:10.346671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf290c-2f6e-4097-a2a6-7d49019f07cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:10.347449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf290c-2f6e-4097-a2a6-7d49019f07cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:10.348164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf290c-2f6e-4097-a2a6-7d49019f07cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:10.349777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf290c-2f6e-4097-a2a6-7d49019f07cf admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:10.359746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1354/2721] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:10.364316 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1c970-337c-484d-95e3-40b3665d5691 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:10.374214 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc1c970-337c-484d-95e3-40b3665d5691 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:10.378688 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2acc3fd-06bb-43e2-9215-9a43dc2d084e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:10.379313 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acc3fd-06bb-43e2-9215-9a43dc2d084e admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:10.379586 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acc3fd-06bb-43e2-9215-9a43dc2d084e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:10.379851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2acc3fd-06bb-43e2-9215-9a43dc2d084e admin admin] PATH_INFO: `/v3/projects/c3c19a830b3b4737b11ddb49d2e46116/users/a81e722427d84e56be2be5792176296a/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:10.393757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2acc3fd-06bb-43e2-9215-9a43dc2d084e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c19a830b3b4737b11ddb49d2e46116, user_id=a81e722427d84e56be2be5792176296a, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:10.396351 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2acc3fd-06bb-43e2-9215-9a43dc2d084e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:10.408590 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1368/2722] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:10 2022] PUT /identity/v3/projects/c3c19a830b3b4737b11ddb49d2e46116/users/a81e722427d84e56be2be5792176296a/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:10.419201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed470-4e7e-4270-a211-51be472e932e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:10.419599 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed470-4e7e-4270-a211-51be472e932e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:10.419886 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed470-4e7e-4270-a211-51be472e932e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:10.447076 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ad6ed470-4e7e-4270-a211-51be472e932e None None] MFA Rules not processed for user `a81e722427d84e56be2be5792176296a`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:10.448281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ad6ed470-4e7e-4270-a211-51be472e932e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:10.490696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:10.514410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1355/2723] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:10.518917 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2acc3fd-06bb-43e2-9215-9a43dc2d084e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:10.575737 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:10.579336 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2acc3fd-06bb-43e2-9215-9a43dc2d084e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:10.585278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895e9c7-3fe3-4978-a2ea-1bce8506a8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:10.586040 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895e9c7-3fe3-4978-a2ea-1bce8506a8cd admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:10.586483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895e9c7-3fe3-4978-a2ea-1bce8506a8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:10.586935 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c895e9c7-3fe3-4978-a2ea-1bce8506a8cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:10.590605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c895e9c7-3fe3-4978-a2ea-1bce8506a8cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:10.590605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c895e9c7-3fe3-4978-a2ea-1bce8506a8cd admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:10.617531 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1369/2724] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:10 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:10.621344 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ed470-4e7e-4270-a211-51be472e932e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:10.631784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ed470-4e7e-4270-a211-51be472e932e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:10.689098 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd620cc-ea22-4584-a464-e08185e2e785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:10.689721 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd620cc-ea22-4584-a464-e08185e2e785 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:10.689996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd620cc-ea22-4584-a464-e08185e2e785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:10.690261 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd620cc-ea22-4584-a464-e08185e2e785 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:10.690912 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd620cc-ea22-4584-a464-e08185e2e785 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:10.692799 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd620cc-ea22-4584-a464-e08185e2e785 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:10.714384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1356/2725] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:10 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:10.718460 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895e9c7-3fe3-4978-a2ea-1bce8506a8cd admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:10.729364 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c895e9c7-3fe3-4978-a2ea-1bce8506a8cd admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:10.733869 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908bb6a-b442-43db-a936-f0b82aeb9b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:10.734454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908bb6a-b442-43db-a936-f0b82aeb9b3f admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:10.734731 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908bb6a-b442-43db-a936-f0b82aeb9b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:10.735002 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3908bb6a-b442-43db-a936-f0b82aeb9b3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:10.735631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908bb6a-b442-43db-a936-f0b82aeb9b3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:10.737215 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3908bb6a-b442-43db-a936-f0b82aeb9b3f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:10.747578 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1370/2726] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:10.751299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd620cc-ea22-4584-a464-e08185e2e785 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:10.761540 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd620cc-ea22-4584-a464-e08185e2e785 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:10.766302 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14da5873-a835-4650-b5c6-403af96cc3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:10.766883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da5873-a835-4650-b5c6-403af96cc3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:10.767157 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da5873-a835-4650-b5c6-403af96cc3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:10.767418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14da5873-a835-4650-b5c6-403af96cc3b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:10.768069 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da5873-a835-4650-b5c6-403af96cc3b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:10.770147 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14da5873-a835-4650-b5c6-403af96cc3b1 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:10.780071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1357/2727] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:10.784180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908bb6a-b442-43db-a936-f0b82aeb9b3f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:10.794805 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3908bb6a-b442-43db-a936-f0b82aeb9b3f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:10.799383 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e68394-6c34-4764-a912-20405d104e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:10.800029 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e68394-6c34-4764-a912-20405d104e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:10.800301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e68394-6c34-4764-a912-20405d104e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:10.800572 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e68394-6c34-4764-a912-20405d104e35 admin admin] PATH_INFO: `/v3/projects/1838684192804c888e4512f72d32516c/users/2e53a3b27db043ff945c00a7fb29a7e2/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:10.814770 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e68394-6c34-4764-a912-20405d104e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1838684192804c888e4512f72d32516c, user_id=2e53a3b27db043ff945c00a7fb29a7e2, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:10.817419 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e68394-6c34-4764-a912-20405d104e35 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:10.829916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1371/2728] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:10 2022] PUT /identity/v3/projects/1838684192804c888e4512f72d32516c/users/2e53a3b27db043ff945c00a7fb29a7e2/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:24:10.840672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4cfe7-8d83-4da6-9c52-f37cf374cfac None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:10.840980 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4cfe7-8d83-4da6-9c52-f37cf374cfac None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:10.841288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df4cfe7-8d83-4da6-9c52-f37cf374cfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:10.869529 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-7df4cfe7-8d83-4da6-9c52-f37cf374cfac None None] MFA Rules not processed for user `2e53a3b27db043ff945c00a7fb29a7e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:10.870670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7df4cfe7-8d83-4da6-9c52-f37cf374cfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:10.932863 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1358/2729] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:10.937355 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e68394-6c34-4764-a912-20405d104e35 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:10.997544 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e68394-6c34-4764-a912-20405d104e35 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:11.002142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e1627-703e-469f-ab7e-c0b836668676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:11.002727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e1627-703e-469f-ab7e-c0b836668676 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:11.003000 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e1627-703e-469f-ab7e-c0b836668676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:11.003780 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e1627-703e-469f-ab7e-c0b836668676 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:11.004383 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e1627-703e-469f-ab7e-c0b836668676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:11.006278 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e1627-703e-469f-ab7e-c0b836668676 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:11.033260 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1372/2730] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:10 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:11.037353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4cfe7-8d83-4da6-9c52-f37cf374cfac None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:11.047576 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df4cfe7-8d83-4da6-9c52-f37cf374cfac None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:11.092643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:11.103784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fd669-5466-4f3f-8aa5-67c8c085f106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:11.104381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fd669-5466-4f3f-8aa5-67c8c085f106 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:11.104658 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fd669-5466-4f3f-8aa5-67c8c085f106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:11.104927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98fd669-5466-4f3f-8aa5-67c8c085f106 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:11.105609 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fd669-5466-4f3f-8aa5-67c8c085f106 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:11.107555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98fd669-5466-4f3f-8aa5-67c8c085f106 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:11.127816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1359/2731] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:11 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:11.132233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e1627-703e-469f-ab7e-c0b836668676 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:11.142565 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e1627-703e-469f-ab7e-c0b836668676 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:11.147842 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41903f-1b50-497f-8f37-e37b437ab05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:11.148428 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41903f-1b50-497f-8f37-e37b437ab05b admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:11.148703 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41903f-1b50-497f-8f37-e37b437ab05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:11.148972 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b41903f-1b50-497f-8f37-e37b437ab05b admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:11.149718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41903f-1b50-497f-8f37-e37b437ab05b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:11.151284 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b41903f-1b50-497f-8f37-e37b437ab05b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:11.161660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1373/2732] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:11.165720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fd669-5466-4f3f-8aa5-67c8c085f106 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:11.175526 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98fd669-5466-4f3f-8aa5-67c8c085f106 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:11.179990 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86923c-a1c2-4ee6-b932-df4c767bf29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:11.180617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86923c-a1c2-4ee6-b932-df4c767bf29d admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:11.181013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86923c-a1c2-4ee6-b932-df4c767bf29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:11.181510 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c86923c-a1c2-4ee6-b932-df4c767bf29d admin admin] PATH_INFO: `/v3/projects/a42bfa5dcd2847e79b4d3290faa22499/users/87c2499cab62452ca24f050ed17a3aec/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:11.196381 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86923c-a1c2-4ee6-b932-df4c767bf29d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42bfa5dcd2847e79b4d3290faa22499, user_id=87c2499cab62452ca24f050ed17a3aec, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:11.199025 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c86923c-a1c2-4ee6-b932-df4c767bf29d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:11.212765 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1360/2733] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:11 2022] PUT /identity/v3/projects/a42bfa5dcd2847e79b4d3290faa22499/users/87c2499cab62452ca24f050ed17a3aec/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:11.216501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41903f-1b50-497f-8f37-e37b437ab05b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:11.226901 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b41903f-1b50-497f-8f37-e37b437ab05b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:11.292162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5745cf7-3253-4cee-a771-dc48b1f8b774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:11.292756 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5745cf7-3253-4cee-a771-dc48b1f8b774 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:11.293031 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5745cf7-3253-4cee-a771-dc48b1f8b774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:11.293334 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5745cf7-3253-4cee-a771-dc48b1f8b774 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:11.293960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5745cf7-3253-4cee-a771-dc48b1f8b774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:11.295527 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5745cf7-3253-4cee-a771-dc48b1f8b774 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:11.308182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1374/2734] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:11.312353 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86923c-a1c2-4ee6-b932-df4c767bf29d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:11.323021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c86923c-a1c2-4ee6-b932-df4c767bf29d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:11.327624 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f41db7-9d9f-415f-9773-71234f607dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:11.328267 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f41db7-9d9f-415f-9773-71234f607dd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:11.328542 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f41db7-9d9f-415f-9773-71234f607dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:11.328812 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f41db7-9d9f-415f-9773-71234f607dd2 admin admin] PATH_INFO: `/v3/projects/a42bfa5dcd2847e79b4d3290faa22499/users/87c2499cab62452ca24f050ed17a3aec/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:11.330896 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f41db7-9d9f-415f-9773-71234f607dd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a42bfa5dcd2847e79b4d3290faa22499, user_id=87c2499cab62452ca24f050ed17a3aec, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:11.334133 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f41db7-9d9f-415f-9773-71234f607dd2 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:11.344544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1361/2735] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:11 2022] PUT /identity/v3/projects/a42bfa5dcd2847e79b4d3290faa22499/users/87c2499cab62452ca24f050ed17a3aec/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:11.409657 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869276b-dcb2-47b6-bee5-b0d14436e6f8 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:11.409854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869276b-dcb2-47b6-bee5-b0d14436e6f8 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:11.410151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8869276b-dcb2-47b6-bee5-b0d14436e6f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:11.437681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8869276b-dcb2-47b6-bee5-b0d14436e6f8 None None] MFA Rules not processed for user `87c2499cab62452ca24f050ed17a3aec`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:11.438880 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8869276b-dcb2-47b6-bee5-b0d14436e6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:11.444483 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:11.509309 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1375/2736] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:11.520370 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f41db7-9d9f-415f-9773-71234f607dd2 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:11.522068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-51f41db7-9d9f-415f-9773-71234f607dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:11.578616 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f41db7-9d9f-415f-9773-71234f607dd2 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:11.582348 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94570a40-14a6-4f0c-8965-94d5ea09db53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:11.582937 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94570a40-14a6-4f0c-8965-94d5ea09db53 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:11.583208 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94570a40-14a6-4f0c-8965-94d5ea09db53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:11.583473 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94570a40-14a6-4f0c-8965-94d5ea09db53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:11.638224 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94570a40-14a6-4f0c-8965-94d5ea09db53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:11.639676 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94570a40-14a6-4f0c-8965-94d5ea09db53 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:11.649468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1362/2737] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:13.309845 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869276b-dcb2-47b6-bee5-b0d14436e6f8 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:13.311340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8869276b-dcb2-47b6-bee5-b0d14436e6f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:13.323813 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8869276b-dcb2-47b6-bee5-b0d14436e6f8 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:13.327417 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91696aac-b592-4869-91e5-3aa9d660da02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:13.328010 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91696aac-b592-4869-91e5-3aa9d660da02 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:13.328317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91696aac-b592-4869-91e5-3aa9d660da02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:13.328592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91696aac-b592-4869-91e5-3aa9d660da02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:13.385610 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91696aac-b592-4869-91e5-3aa9d660da02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:13.387026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91696aac-b592-4869-91e5-3aa9d660da02 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:13.395872 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1376/2738] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:13.917869 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94570a40-14a6-4f0c-8965-94d5ea09db53 service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:13.927062 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94570a40-14a6-4f0c-8965-94d5ea09db53 service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:13.930691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434e5da1-c08d-4ed1-bf44-1906b3997ef5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:13.931282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e5da1-c08d-4ed1-bf44-1906b3997ef5 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:13.931556 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e5da1-c08d-4ed1-bf44-1906b3997ef5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:13.931825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434e5da1-c08d-4ed1-bf44-1906b3997ef5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:13.940670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e5da1-c08d-4ed1-bf44-1906b3997ef5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:13.942105 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434e5da1-c08d-4ed1-bf44-1906b3997ef5 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:13.951442 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1363/2739] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:17.818352 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91696aac-b592-4869-91e5-3aa9d660da02 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:17.881154 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91696aac-b592-4869-91e5-3aa9d660da02 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:17.887478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592b4c3-9654-4232-8196-105ccb2a8592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:17.888269 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592b4c3-9654-4232-8196-105ccb2a8592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:17.888546 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592b4c3-9654-4232-8196-105ccb2a8592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:17.888815 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592b4c3-9654-4232-8196-105ccb2a8592 admin admin] PATH_INFO: `/v3/users/a81e722427d84e56be2be5792176296a` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:17.889952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592b4c3-9654-4232-8196-105ccb2a8592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a81e722427d84e56be2be5792176296a)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:17.891849 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592b4c3-9654-4232-8196-105ccb2a8592 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:17.933936 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:17.936617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b592b4c3-9654-4232-8196-105ccb2a8592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a81e722427d84e56be2be5792176296a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:17.944221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b592b4c3-9654-4232-8196-105ccb2a8592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a81e722427d84e56be2be5792176296a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:17.951975 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b592b4c3-9654-4232-8196-105ccb2a8592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a81e722427d84e56be2be5792176296a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:17.952580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b592b4c3-9654-4232-8196-105ccb2a8592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a81e722427d84e56be2be5792176296a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:17.964430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-b592b4c3-9654-4232-8196-105ccb2a8592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a81e722427d84e56be2be5792176296a'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:17.970757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1377/2740] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:24:17 2022] DELETE /identity/v3/users/a81e722427d84e56be2be5792176296a => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:17.975058 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db01fc-2df9-429a-a3f1-97a4f6cbf432 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:17.975553 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db01fc-2df9-429a-a3f1-97a4f6cbf432 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:17.975711 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db01fc-2df9-429a-a3f1-97a4f6cbf432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:17.995696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-00db01fc-2df9-429a-a3f1-97a4f6cbf432 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:17.996889 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-00db01fc-2df9-429a-a3f1-97a4f6cbf432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:18.035504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:18.055466 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1364/2741] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:18.065381 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592b4c3-9654-4232-8196-105ccb2a8592 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:18.066764 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b592b4c3-9654-4232-8196-105ccb2a8592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:18.123739 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592b4c3-9654-4232-8196-105ccb2a8592 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:18.127639 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac43e4-969a-47a0-b095-cecc97b740ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:18.128238 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac43e4-969a-47a0-b095-cecc97b740ae service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:18.128514 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac43e4-969a-47a0-b095-cecc97b740ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:18.128784 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac43e4-969a-47a0-b095-cecc97b740ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:18.137717 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac43e4-969a-47a0-b095-cecc97b740ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:18.139401 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac43e4-969a-47a0-b095-cecc97b740ae service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:18.148604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1378/2742] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:18.434925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db01fc-2df9-429a-a3f1-97a4f6cbf432 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:18.436447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-00db01fc-2df9-429a-a3f1-97a4f6cbf432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:18.451832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db01fc-2df9-429a-a3f1-97a4f6cbf432 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:18.456485 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:18.457121 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:18.457446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:18.457749 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea admin admin] PATH_INFO: `/v3/projects/c3c19a830b3b4737b11ddb49d2e46116` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:18.458790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3c19a830b3b4737b11ddb49d2e46116)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:18.461388 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:18.506460 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3c19a830b3b4737b11ddb49d2e46116', 'request_id': 'req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:18.513269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3c19a830b3b4737b11ddb49d2e46116', 'request_id': 'req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:18.518648 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3c19a830b3b4737b11ddb49d2e46116', 'request_id': 'req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:18.529769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea admin admin] The token cache is being invalidate because project c3c19a830b3b4737b11ddb49d2e46116 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:24:18.530050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:18.530668 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:18.532113 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1365/2743] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:24:18 2022] DELETE /identity/v3/projects/c3c19a830b3b4737b11ddb49d2e46116 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:18.536047 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac43e4-969a-47a0-b095-cecc97b740ae service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:18.546361 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac43e4-969a-47a0-b095-cecc97b740ae service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:18.602371 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48a323-0d47-4f30-890f-2f357c69518e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:18.602995 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48a323-0d47-4f30-890f-2f357c69518e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:18.603302 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48a323-0d47-4f30-890f-2f357c69518e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:18.603575 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48a323-0d47-4f30-890f-2f357c69518e admin admin] PATH_INFO: `/v3/users/2e53a3b27db043ff945c00a7fb29a7e2` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:18.604680 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48a323-0d47-4f30-890f-2f357c69518e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e53a3b27db043ff945c00a7fb29a7e2)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:18.606622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf48a323-0d47-4f30-890f-2f357c69518e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:18.652365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-cf48a323-0d47-4f30-890f-2f357c69518e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e53a3b27db043ff945c00a7fb29a7e2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:18.659023 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-cf48a323-0d47-4f30-890f-2f357c69518e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e53a3b27db043ff945c00a7fb29a7e2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:18.667114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-cf48a323-0d47-4f30-890f-2f357c69518e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e53a3b27db043ff945c00a7fb29a7e2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:18.667739 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-cf48a323-0d47-4f30-890f-2f357c69518e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e53a3b27db043ff945c00a7fb29a7e2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:18.669702 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:18.682647 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-cf48a323-0d47-4f30-890f-2f357c69518e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e53a3b27db043ff945c00a7fb29a7e2'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:18.688553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1379/2744] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:24:18 2022] DELETE /identity/v3/users/2e53a3b27db043ff945c00a7fb29a7e2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:18.968985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:18.970740 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:18.984170 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bdfa5b-8a8e-420f-9206-a24f9d94e1ea admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:19.041124 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:19.041762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:19.042033 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:19.042297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5 admin admin] PATH_INFO: `/v3/projects/1838684192804c888e4512f72d32516c` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:19.043410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1838684192804c888e4512f72d32516c)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:19.045323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:19.089908 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1838684192804c888e4512f72d32516c', 'request_id': 'req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:19.096998 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1838684192804c888e4512f72d32516c', 'request_id': 'req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:19.102501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1838684192804c888e4512f72d32516c', 'request_id': 'req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:19.109340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:19.114164 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5 admin admin] The token cache is being invalidate because project 1838684192804c888e4512f72d32516c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:24:19.114474 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:19.115446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:19.117334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1366/2745] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:24:18 2022] DELETE /identity/v3/projects/1838684192804c888e4512f72d32516c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:19.121428 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48a323-0d47-4f30-890f-2f357c69518e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:19.122827 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-cf48a323-0d47-4f30-890f-2f357c69518e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:19.135850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48a323-0d47-4f30-890f-2f357c69518e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:19.193241 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abeb223-060d-4eb1-8547-b6753276db93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:19.193806 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abeb223-060d-4eb1-8547-b6753276db93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:19.194081 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abeb223-060d-4eb1-8547-b6753276db93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:19.194373 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abeb223-060d-4eb1-8547-b6753276db93 admin admin] PATH_INFO: `/v3/users/87c2499cab62452ca24f050ed17a3aec` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:19.196106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abeb223-060d-4eb1-8547-b6753276db93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87c2499cab62452ca24f050ed17a3aec)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:19.197951 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abeb223-060d-4eb1-8547-b6753276db93 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:19.243077 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2abeb223-060d-4eb1-8547-b6753276db93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87c2499cab62452ca24f050ed17a3aec'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:19.249819 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2abeb223-060d-4eb1-8547-b6753276db93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87c2499cab62452ca24f050ed17a3aec'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:19.257654 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2abeb223-060d-4eb1-8547-b6753276db93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87c2499cab62452ca24f050ed17a3aec'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:19.258315 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2abeb223-060d-4eb1-8547-b6753276db93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87c2499cab62452ca24f050ed17a3aec'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:19.271786 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2abeb223-060d-4eb1-8547-b6753276db93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87c2499cab62452ca24f050ed17a3aec'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:19.277661 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1380/2746] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:24:19 2022] DELETE /identity/v3/users/87c2499cab62452ca24f050ed17a3aec => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:24:19.551143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:19.552656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:19.566509 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8c3d0-9dd9-407e-8c8c-3b4ebc7e8dd5 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:19.624246 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8890d53c-0c1b-4ae1-a841-8e5754068d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:19.625640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890d53c-0c1b-4ae1-a841-8e5754068d9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:19.626684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890d53c-0c1b-4ae1-a841-8e5754068d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:19.627449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8890d53c-0c1b-4ae1-a841-8e5754068d9f admin admin] PATH_INFO: `/v3/projects/a42bfa5dcd2847e79b4d3290faa22499` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:19.629983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890d53c-0c1b-4ae1-a841-8e5754068d9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a42bfa5dcd2847e79b4d3290faa22499)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:19.633776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8890d53c-0c1b-4ae1-a841-8e5754068d9f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:19.682703 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8890d53c-0c1b-4ae1-a841-8e5754068d9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a42bfa5dcd2847e79b4d3290faa22499', 'request_id': 'req-8890d53c-0c1b-4ae1-a841-8e5754068d9f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:19.689881 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8890d53c-0c1b-4ae1-a841-8e5754068d9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a42bfa5dcd2847e79b4d3290faa22499', 'request_id': 'req-8890d53c-0c1b-4ae1-a841-8e5754068d9f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:19.695428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8890d53c-0c1b-4ae1-a841-8e5754068d9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a42bfa5dcd2847e79b4d3290faa22499', 'request_id': 'req-8890d53c-0c1b-4ae1-a841-8e5754068d9f'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:19.706505 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8890d53c-0c1b-4ae1-a841-8e5754068d9f admin admin] The token cache is being invalidate because project a42bfa5dcd2847e79b4d3290faa22499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:24:19.706794 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8890d53c-0c1b-4ae1-a841-8e5754068d9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:19.707453 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-8890d53c-0c1b-4ae1-a841-8e5754068d9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:19.708969 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1367/2747] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:24:19 2022] DELETE /identity/v3/projects/a42bfa5dcd2847e79b4d3290faa22499 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:19.713816 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ce73e-066a-485a-af7c-7dcff73f9e40 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:19.714103 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ce73e-066a-485a-af7c-7dcff73f9e40 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:19.714378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ce73e-066a-485a-af7c-7dcff73f9e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:19.734918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-631ce73e-066a-485a-af7c-7dcff73f9e40 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:19.736053 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-631ce73e-066a-485a-af7c-7dcff73f9e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:19.795771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1381/2748] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:19.803786 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdae5c-6e4b-4328-9359-3c401e39787b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:19.804704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdae5c-6e4b-4328-9359-3c401e39787b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:19.804967 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bdae5c-6e4b-4328-9359-3c401e39787b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:19.824899 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-22bdae5c-6e4b-4328-9359-3c401e39787b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:19.826149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-22bdae5c-6e4b-4328-9359-3c401e39787b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:19.833984 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1368/2749] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:19.838766 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ce73e-066a-485a-af7c-7dcff73f9e40 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:19.843619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:19.850177 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ce73e-066a-485a-af7c-7dcff73f9e40 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:19.856133 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92232edc-9606-4a02-bc82-2f4d7b362065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:19.856728 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92232edc-9606-4a02-bc82-2f4d7b362065 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:19.857007 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92232edc-9606-4a02-bc82-2f4d7b362065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:19.857317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92232edc-9606-4a02-bc82-2f4d7b362065 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:19.857955 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92232edc-9606-4a02-bc82-2f4d7b362065 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:24:19.858254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92232edc-9606-4a02-bc82-2f4d7b362065 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:19.859826 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92232edc-9606-4a02-bc82-2f4d7b362065 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:19.871336 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1382/2750] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:24:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:24:19.875553 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdae5c-6e4b-4328-9359-3c401e39787b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:19.885457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bdae5c-6e4b-4328-9359-3c401e39787b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:19.890061 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98d06f-be8f-426e-b812-8775054e56f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:19.890669 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98d06f-be8f-426e-b812-8775054e56f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:19.890946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98d06f-be8f-426e-b812-8775054e56f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:19.891215 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc98d06f-be8f-426e-b812-8775054e56f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:19.891832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98d06f-be8f-426e-b812-8775054e56f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:19.893736 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc98d06f-be8f-426e-b812-8775054e56f5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:19.920610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1369/2751] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:19 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:19.924553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92232edc-9606-4a02-bc82-2f4d7b362065 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:19.935392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92232edc-9606-4a02-bc82-2f4d7b362065 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:19.993180 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02d75d-19ab-4d3b-a96f-5e1a86790f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:19.993771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02d75d-19ab-4d3b-a96f-5e1a86790f26 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:19.994049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02d75d-19ab-4d3b-a96f-5e1a86790f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:19.994316 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02d75d-19ab-4d3b-a96f-5e1a86790f26 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:19.994946 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02d75d-19ab-4d3b-a96f-5e1a86790f26 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:19.996866 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f02d75d-19ab-4d3b-a96f-5e1a86790f26 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:20.018175 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1383/2752] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:19 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:20.022437 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98d06f-be8f-426e-b812-8775054e56f5 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:20.032693 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc98d06f-be8f-426e-b812-8775054e56f5 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:20.037328 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e6c4b-2235-4913-a139-2b6910637bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:20.037916 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e6c4b-2235-4913-a139-2b6910637bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:20.038195 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e6c4b-2235-4913-a139-2b6910637bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:20.038463 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e6c4b-2235-4913-a139-2b6910637bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:20.039091 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e6c4b-2235-4913-a139-2b6910637bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:20.040662 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2e6c4b-2235-4913-a139-2b6910637bc6 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:20.050369 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1370/2753] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:20.054612 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02d75d-19ab-4d3b-a96f-5e1a86790f26 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:20.064415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f02d75d-19ab-4d3b-a96f-5e1a86790f26 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:20.068952 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fa2b65-c6a4-4caf-a261-45b8585b39c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:20.070305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa2b65-c6a4-4caf-a261-45b8585b39c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:20.070582 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa2b65-c6a4-4caf-a261-45b8585b39c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:20.070850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fa2b65-c6a4-4caf-a261-45b8585b39c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:20.071478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa2b65-c6a4-4caf-a261-45b8585b39c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:20.073044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fa2b65-c6a4-4caf-a261-45b8585b39c4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:20.083405 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1384/2754] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:20.087507 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e6c4b-2235-4913-a139-2b6910637bc6 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:20.097904 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2e6c4b-2235-4913-a139-2b6910637bc6 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:20.102545 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04091e84-c724-4bc2-a85a-76a2f38e2515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:20.103178 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04091e84-c724-4bc2-a85a-76a2f38e2515 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:20.103456 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04091e84-c724-4bc2-a85a-76a2f38e2515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:20.103725 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04091e84-c724-4bc2-a85a-76a2f38e2515 admin admin] PATH_INFO: `/v3/projects/c24fadf13699459bad2088cf0c086283/users/abec99712c9e4d14ab8c29b18c79419b/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:20.119059 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04091e84-c724-4bc2-a85a-76a2f38e2515 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c24fadf13699459bad2088cf0c086283, user_id=abec99712c9e4d14ab8c29b18c79419b, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:20.121731 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04091e84-c724-4bc2-a85a-76a2f38e2515 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:20.135509 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1371/2755] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:20 2022] PUT /identity/v3/projects/c24fadf13699459bad2088cf0c086283/users/abec99712c9e4d14ab8c29b18c79419b/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:20.147263 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ffb30-650b-44a2-a739-a61815821150 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:20.147769 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ffb30-650b-44a2-a739-a61815821150 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:20.147909 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ffb30-650b-44a2-a739-a61815821150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:20.176142 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-b84ffb30-650b-44a2-a739-a61815821150 None None] MFA Rules not processed for user `abec99712c9e4d14ab8c29b18c79419b`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:20.177378 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-b84ffb30-650b-44a2-a739-a61815821150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:20.242803 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1385/2756] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:20.247325 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04091e84-c724-4bc2-a85a-76a2f38e2515 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:20.274074 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:20.311760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04091e84-c724-4bc2-a85a-76a2f38e2515 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:20.316349 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d3e60-a266-479a-8761-6f023a39d2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:20.316948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d3e60-a266-479a-8761-6f023a39d2f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:20.317253 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d3e60-a266-479a-8761-6f023a39d2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:20.317530 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d3e60-a266-479a-8761-6f023a39d2f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:20.318158 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d3e60-a266-479a-8761-6f023a39d2f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:20.320043 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d3e60-a266-479a-8761-6f023a39d2f3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:20.345378 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1372/2757] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:20 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:20.349427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ffb30-650b-44a2-a739-a61815821150 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:20.360734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ffb30-650b-44a2-a739-a61815821150 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:20.409622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:20.417021 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee96a4a-4c15-4501-b270-8c21bfeedf0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:20.417636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee96a4a-4c15-4501-b270-8c21bfeedf0e admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:20.417913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee96a4a-4c15-4501-b270-8c21bfeedf0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:20.418201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee96a4a-4c15-4501-b270-8c21bfeedf0e admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:20.418808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee96a4a-4c15-4501-b270-8c21bfeedf0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:20.420720 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee96a4a-4c15-4501-b270-8c21bfeedf0e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:20.441638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1386/2758] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:20 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:20.445845 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d3e60-a266-479a-8761-6f023a39d2f3 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:20.456226 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d3e60-a266-479a-8761-6f023a39d2f3 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:20.460840 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b05db-c775-408e-9702-103f0d16cc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:20.461435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b05db-c775-408e-9702-103f0d16cc85 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:20.461785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b05db-c775-408e-9702-103f0d16cc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:20.462055 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b05db-c775-408e-9702-103f0d16cc85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:20.462680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b05db-c775-408e-9702-103f0d16cc85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:20.464269 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b05db-c775-408e-9702-103f0d16cc85 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:20.475161 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1373/2759] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:20.479734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee96a4a-4c15-4501-b270-8c21bfeedf0e admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:20.489903 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee96a4a-4c15-4501-b270-8c21bfeedf0e admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:20.495468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6effd8-31f0-4cad-bb98-555109c82bc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:20.496100 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6effd8-31f0-4cad-bb98-555109c82bc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:20.496421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6effd8-31f0-4cad-bb98-555109c82bc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:20.496733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6effd8-31f0-4cad-bb98-555109c82bc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:20.497509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6effd8-31f0-4cad-bb98-555109c82bc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:20.499220 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6effd8-31f0-4cad-bb98-555109c82bc1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:20.509615 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1387/2760] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:20.513650 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b05db-c775-408e-9702-103f0d16cc85 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:20.523475 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b05db-c775-408e-9702-103f0d16cc85 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:20.528142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cebdc7-0b8a-470c-95fe-2e356e89e2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:20.528771 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cebdc7-0b8a-470c-95fe-2e356e89e2b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:20.529044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cebdc7-0b8a-470c-95fe-2e356e89e2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:20.529343 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cebdc7-0b8a-470c-95fe-2e356e89e2b7 admin admin] PATH_INFO: `/v3/projects/e4f1d727a9d14cb3b7500bc9cae224e8/users/a294eda1b1484d9d8db582eaf6f2d820/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:20.543966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cebdc7-0b8a-470c-95fe-2e356e89e2b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4f1d727a9d14cb3b7500bc9cae224e8, user_id=a294eda1b1484d9d8db582eaf6f2d820, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:20.546606 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cebdc7-0b8a-470c-95fe-2e356e89e2b7 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:20.560018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1374/2761] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:20 2022] PUT /identity/v3/projects/e4f1d727a9d14cb3b7500bc9cae224e8/users/a294eda1b1484d9d8db582eaf6f2d820/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:20.570122 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28434836-6bb9-4042-ab4b-27109f30c329 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:20.570401 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28434836-6bb9-4042-ab4b-27109f30c329 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:20.570672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28434836-6bb9-4042-ab4b-27109f30c329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:20.597825 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-28434836-6bb9-4042-ab4b-27109f30c329 None None] MFA Rules not processed for user `a294eda1b1484d9d8db582eaf6f2d820`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:20.599015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-28434836-6bb9-4042-ab4b-27109f30c329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:20.662638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1388/2762] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:20.667707 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cebdc7-0b8a-470c-95fe-2e356e89e2b7 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:20.729660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cebdc7-0b8a-470c-95fe-2e356e89e2b7 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:20.734235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dba9a0-876f-4e0d-8af1-829f016e9d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:20.734825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dba9a0-876f-4e0d-8af1-829f016e9d63 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:20.735096 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dba9a0-876f-4e0d-8af1-829f016e9d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:20.735362 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dba9a0-876f-4e0d-8af1-829f016e9d63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:20.735975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dba9a0-876f-4e0d-8af1-829f016e9d63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:20.737939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dba9a0-876f-4e0d-8af1-829f016e9d63 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:20.763841 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1375/2763] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:24:20 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:20.768015 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28434836-6bb9-4042-ab4b-27109f30c329 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:20.779387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28434836-6bb9-4042-ab4b-27109f30c329 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:20.834631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f0bd1-3bbb-4fbc-a59b-41aca4e52668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:20.836341 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f0bd1-3bbb-4fbc-a59b-41aca4e52668 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:20.836629 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f0bd1-3bbb-4fbc-a59b-41aca4e52668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:20.836904 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f0bd1-3bbb-4fbc-a59b-41aca4e52668 admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:20.837597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f0bd1-3bbb-4fbc-a59b-41aca4e52668 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:20.839530 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255f0bd1-3bbb-4fbc-a59b-41aca4e52668 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:20.861503 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1389/2764] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:24:20 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:24:20.868198 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dba9a0-876f-4e0d-8af1-829f016e9d63 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:20.878302 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dba9a0-876f-4e0d-8af1-829f016e9d63 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:20.882901 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe666e-1741-4833-a07c-24f633f5238a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:20.883621 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe666e-1741-4833-a07c-24f633f5238a admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:20.883927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe666e-1741-4833-a07c-24f633f5238a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:20.884198 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe666e-1741-4833-a07c-24f633f5238a admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:20.884901 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe666e-1741-4833-a07c-24f633f5238a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:20.886513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe666e-1741-4833-a07c-24f633f5238a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:20.896776 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1376/2765] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:20.900924 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f0bd1-3bbb-4fbc-a59b-41aca4e52668 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:20.911588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255f0bd1-3bbb-4fbc-a59b-41aca4e52668 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:20.916248 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04857d3-284b-41f9-acc3-385500f4462b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:20.916925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04857d3-284b-41f9-acc3-385500f4462b admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:20.917233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04857d3-284b-41f9-acc3-385500f4462b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:20.917512 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04857d3-284b-41f9-acc3-385500f4462b admin admin] PATH_INFO: `/v3/projects/102645d12dbc4e3f8a91c56af17ee6e2/users/4889a488ec2543fb87b6594a35a7079d/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:20.932370 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04857d3-284b-41f9-acc3-385500f4462b admin admin] RBAC: Authorizing `identity:create_grant(project_id=102645d12dbc4e3f8a91c56af17ee6e2, user_id=4889a488ec2543fb87b6594a35a7079d, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:20.935039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04857d3-284b-41f9-acc3-385500f4462b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:20.947597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1390/2766] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:20 2022] PUT /identity/v3/projects/102645d12dbc4e3f8a91c56af17ee6e2/users/4889a488ec2543fb87b6594a35a7079d/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:20.952013 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe666e-1741-4833-a07c-24f633f5238a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:20.961959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe666e-1741-4833-a07c-24f633f5238a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:21.006602 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:21.019439 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5e8db9-4f6d-4a15-b596-11d06ab86ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:21.019971 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e8db9-4f6d-4a15-b596-11d06ab86ddd admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:21.020244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e8db9-4f6d-4a15-b596-11d06ab86ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:21.020512 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5e8db9-4f6d-4a15-b596-11d06ab86ddd admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:21.021135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e8db9-4f6d-4a15-b596-11d06ab86ddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:21.022730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5e8db9-4f6d-4a15-b596-11d06ab86ddd admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:21.032914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1377/2767] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:24:21.037030 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04857d3-284b-41f9-acc3-385500f4462b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:21.047659 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04857d3-284b-41f9-acc3-385500f4462b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:21.052298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e81509-b904-4c29-bdde-7b92690720a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:21.052938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e81509-b904-4c29-bdde-7b92690720a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:21.053246 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e81509-b904-4c29-bdde-7b92690720a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:21.053521 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e81509-b904-4c29-bdde-7b92690720a2 admin admin] PATH_INFO: `/v3/projects/102645d12dbc4e3f8a91c56af17ee6e2/users/4889a488ec2543fb87b6594a35a7079d/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:21.055716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e81509-b904-4c29-bdde-7b92690720a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=102645d12dbc4e3f8a91c56af17ee6e2, user_id=4889a488ec2543fb87b6594a35a7079d, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:21.058385 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e81509-b904-4c29-bdde-7b92690720a2 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:21.068359 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1391/2768] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:24:21 2022] PUT /identity/v3/projects/102645d12dbc4e3f8a91c56af17ee6e2/users/4889a488ec2543fb87b6594a35a7079d/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:21.078982 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16c0d9-68b0-4ced-952c-dbabd37685a7 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:21.079189 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16c0d9-68b0-4ced-952c-dbabd37685a7 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:21.079393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be16c0d9-68b0-4ced-952c-dbabd37685a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:21.107118 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-be16c0d9-68b0-4ced-952c-dbabd37685a7 None None] MFA Rules not processed for user `4889a488ec2543fb87b6594a35a7079d`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:21.108318 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-be16c0d9-68b0-4ced-952c-dbabd37685a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:21.174807 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1378/2769] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:21 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:21.186075 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e81509-b904-4c29-bdde-7b92690720a2 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:21.187871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-d6e81509-b904-4c29-bdde-7b92690720a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:21.208841 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:21.248885 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e81509-b904-4c29-bdde-7b92690720a2 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:21.252500 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aedd80-f452-460a-b718-cf7749dc2d30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:21.253104 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aedd80-f452-460a-b718-cf7749dc2d30 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:21.253413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aedd80-f452-460a-b718-cf7749dc2d30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:21.253685 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aedd80-f452-460a-b718-cf7749dc2d30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:21.311760 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aedd80-f452-460a-b718-cf7749dc2d30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:21.313122 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28aedd80-f452-460a-b718-cf7749dc2d30 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:21.321809 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1392/2770] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:22.817608 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16c0d9-68b0-4ced-952c-dbabd37685a7 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:22.819026 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-be16c0d9-68b0-4ced-952c-dbabd37685a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:22.831501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be16c0d9-68b0-4ced-952c-dbabd37685a7 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:22.835048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1739de0-e6ce-44d9-a8ae-cc4032a1cfde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:22.835633 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1739de0-e6ce-44d9-a8ae-cc4032a1cfde service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:22.835899 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1739de0-e6ce-44d9-a8ae-cc4032a1cfde service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:22.836159 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1739de0-e6ce-44d9-a8ae-cc4032a1cfde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:22.893720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1739de0-e6ce-44d9-a8ae-cc4032a1cfde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:22.895144 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1739de0-e6ce-44d9-a8ae-cc4032a1cfde service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:22.903848 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1379/2771] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:22 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:23.429568 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aedd80-f452-460a-b718-cf7749dc2d30 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:23.438470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aedd80-f452-460a-b718-cf7749dc2d30 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:23.442172 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899317aa-f7a2-47ce-9db2-87544bce679e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:23.442763 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899317aa-f7a2-47ce-9db2-87544bce679e service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:23.443040 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899317aa-f7a2-47ce-9db2-87544bce679e service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:23.443310 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899317aa-f7a2-47ce-9db2-87544bce679e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:23.452846 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899317aa-f7a2-47ce-9db2-87544bce679e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:23.454289 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899317aa-f7a2-47ce-9db2-87544bce679e service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:23.463749 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1393/2772] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:23 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:27.330930 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1739de0-e6ce-44d9-a8ae-cc4032a1cfde service cinder] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:27.393988 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1739de0-e6ce-44d9-a8ae-cc4032a1cfde service cinder] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:27.398558 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb313411-71a7-41b5-8623-e9c2af71bba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:27.399175 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb313411-71a7-41b5-8623-e9c2af71bba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:27.399449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb313411-71a7-41b5-8623-e9c2af71bba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:27.399715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb313411-71a7-41b5-8623-e9c2af71bba8 admin admin] PATH_INFO: `/v3/users/abec99712c9e4d14ab8c29b18c79419b` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:27.400826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb313411-71a7-41b5-8623-e9c2af71bba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abec99712c9e4d14ab8c29b18c79419b)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:27.402800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb313411-71a7-41b5-8623-e9c2af71bba8 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:27.412761 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=100 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:27.447966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eb313411-71a7-41b5-8623-e9c2af71bba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abec99712c9e4d14ab8c29b18c79419b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:27.455396 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eb313411-71a7-41b5-8623-e9c2af71bba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abec99712c9e4d14ab8c29b18c79419b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:27.467325 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eb313411-71a7-41b5-8623-e9c2af71bba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abec99712c9e4d14ab8c29b18c79419b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:27.467985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eb313411-71a7-41b5-8623-e9c2af71bba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abec99712c9e4d14ab8c29b18c79419b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:27.468587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-eb313411-71a7-41b5-8623-e9c2af71bba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abec99712c9e4d14ab8c29b18c79419b'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:27.482188 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1380/2773] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:24:27 2022] DELETE /identity/v3/users/abec99712c9e4d14ab8c29b18c79419b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:27.486642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f871e-b5e8-41b7-9758-50b9a165104b None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:27.487145 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f871e-b5e8-41b7-9758-50b9a165104b None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:27.487252 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8f871e-b5e8-41b7-9758-50b9a165104b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:27.505706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-1e8f871e-b5e8-41b7-9758-50b9a165104b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:27.506895 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1e8f871e-b5e8-41b7-9758-50b9a165104b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:27.564528 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1394/2774] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:27.574104 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb313411-71a7-41b5-8623-e9c2af71bba8 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:27.575611 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-eb313411-71a7-41b5-8623-e9c2af71bba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:27.631839 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb313411-71a7-41b5-8623-e9c2af71bba8 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:27.635461 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17835e4-7365-4164-bda2-ce9288aa966b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:27.636053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17835e4-7365-4164-bda2-ce9288aa966b service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:27.636330 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17835e4-7365-4164-bda2-ce9288aa966b service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:27.636599 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17835e4-7365-4164-bda2-ce9288aa966b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:27.645954 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17835e4-7365-4164-bda2-ce9288aa966b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:27.647366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17835e4-7365-4164-bda2-ce9288aa966b service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:27.657428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1381/2775] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:24:27.925930 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f871e-b5e8-41b7-9758-50b9a165104b None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:27.927455 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1e8f871e-b5e8-41b7-9758-50b9a165104b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:27.939726 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8f871e-b5e8-41b7-9758-50b9a165104b None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:27.944376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33b009-7fc3-4923-9322-3254023358c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:27.945014 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33b009-7fc3-4923-9322-3254023358c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:27.945330 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33b009-7fc3-4923-9322-3254023358c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:27.945604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c33b009-7fc3-4923-9322-3254023358c4 admin admin] PATH_INFO: `/v3/projects/c24fadf13699459bad2088cf0c086283` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:27.946767 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33b009-7fc3-4923-9322-3254023358c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c24fadf13699459bad2088cf0c086283)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:27.948676 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c33b009-7fc3-4923-9322-3254023358c4 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:27.998307 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9c33b009-7fc3-4923-9322-3254023358c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c24fadf13699459bad2088cf0c086283', 'request_id': 'req-9c33b009-7fc3-4923-9322-3254023358c4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.004451 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9c33b009-7fc3-4923-9322-3254023358c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c24fadf13699459bad2088cf0c086283', 'request_id': 'req-9c33b009-7fc3-4923-9322-3254023358c4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.005624 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:28.011887 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9c33b009-7fc3-4923-9322-3254023358c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c24fadf13699459bad2088cf0c086283', 'request_id': 'req-9c33b009-7fc3-4923-9322-3254023358c4'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.025098 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9c33b009-7fc3-4923-9322-3254023358c4 admin admin] The token cache is being invalidate because project c24fadf13699459bad2088cf0c086283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:24:28.025432 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9c33b009-7fc3-4923-9322-3254023358c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.026048 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-9c33b009-7fc3-4923-9322-3254023358c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.027511 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1395/2776] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:24:27 2022] DELETE /identity/v3/projects/c24fadf13699459bad2088cf0c086283 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:28.032337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17835e4-7365-4164-bda2-ce9288aa966b service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:28.041784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17835e4-7365-4164-bda2-ce9288aa966b service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:28.097587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e3a43-392b-48cd-acad-ccb3abb7113b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:28.098202 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e3a43-392b-48cd-acad-ccb3abb7113b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:28.098477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e3a43-392b-48cd-acad-ccb3abb7113b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:28.098744 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20e3a43-392b-48cd-acad-ccb3abb7113b admin admin] PATH_INFO: `/v3/users/a294eda1b1484d9d8db582eaf6f2d820` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:28.099858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e3a43-392b-48cd-acad-ccb3abb7113b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a294eda1b1484d9d8db582eaf6f2d820)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:28.101787 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20e3a43-392b-48cd-acad-ccb3abb7113b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:28.135034 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:28.149129 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c20e3a43-392b-48cd-acad-ccb3abb7113b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a294eda1b1484d9d8db582eaf6f2d820'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.155925 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c20e3a43-392b-48cd-acad-ccb3abb7113b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a294eda1b1484d9d8db582eaf6f2d820'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.167495 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c20e3a43-392b-48cd-acad-ccb3abb7113b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a294eda1b1484d9d8db582eaf6f2d820'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.168161 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c20e3a43-392b-48cd-acad-ccb3abb7113b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a294eda1b1484d9d8db582eaf6f2d820'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.168769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c20e3a43-392b-48cd-acad-ccb3abb7113b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a294eda1b1484d9d8db582eaf6f2d820'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.182826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1382/2777] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:24:28 2022] DELETE /identity/v3/users/a294eda1b1484d9d8db582eaf6f2d820 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:28.454291 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33b009-7fc3-4923-9322-3254023358c4 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:28.455852 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9c33b009-7fc3-4923-9322-3254023358c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:28.468852 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c33b009-7fc3-4923-9322-3254023358c4 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:28.526646 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:28.527325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:28.527532 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:28.527802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0 admin admin] PATH_INFO: `/v3/projects/e4f1d727a9d14cb3b7500bc9cae224e8` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:28.528907 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4f1d727a9d14cb3b7500bc9cae224e8)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:28.530868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:28.576872 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4f1d727a9d14cb3b7500bc9cae224e8', 'request_id': 'req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.583353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4f1d727a9d14cb3b7500bc9cae224e8', 'request_id': 'req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.588865 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4f1d727a9d14cb3b7500bc9cae224e8', 'request_id': 'req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.600730 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0 admin admin] The token cache is being invalidate because project e4f1d727a9d14cb3b7500bc9cae224e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:24:28.601012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.601642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.603087 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1396/2778] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:24:28 2022] DELETE /identity/v3/projects/e4f1d727a9d14cb3b7500bc9cae224e8 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:28.607727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e3a43-392b-48cd-acad-ccb3abb7113b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:28.609206 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-c20e3a43-392b-48cd-acad-ccb3abb7113b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:28.622420 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20e3a43-392b-48cd-acad-ccb3abb7113b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:28.678274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe449a0a-565f-49f1-a723-0bc8856448b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:28.678920 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe449a0a-565f-49f1-a723-0bc8856448b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:28.679196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe449a0a-565f-49f1-a723-0bc8856448b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:28.679464 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe449a0a-565f-49f1-a723-0bc8856448b0 admin admin] PATH_INFO: `/v3/users/4889a488ec2543fb87b6594a35a7079d` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:28.680568 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe449a0a-565f-49f1-a723-0bc8856448b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4889a488ec2543fb87b6594a35a7079d)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:28.682544 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe449a0a-565f-49f1-a723-0bc8856448b0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:28.738454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fe449a0a-565f-49f1-a723-0bc8856448b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4889a488ec2543fb87b6594a35a7079d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.745331 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fe449a0a-565f-49f1-a723-0bc8856448b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4889a488ec2543fb87b6594a35a7079d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.757659 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fe449a0a-565f-49f1-a723-0bc8856448b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4889a488ec2543fb87b6594a35a7079d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.758351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fe449a0a-565f-49f1-a723-0bc8856448b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4889a488ec2543fb87b6594a35a7079d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.758948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-fe449a0a-565f-49f1-a723-0bc8856448b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4889a488ec2543fb87b6594a35a7079d'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:28.771415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1383/2779] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:24:28 2022] DELETE /identity/v3/users/4889a488ec2543fb87b6594a35a7079d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:29.045612 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:24:29.047241 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:29.062534 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9cc8c0-fd9b-4f91-a8f7-c3f18e2e03c0 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:24:29.120259 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3326b-8d25-4b42-b780-11ff04ce1f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:24:29.120886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3326b-8d25-4b42-b780-11ff04ce1f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:29.121165 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3326b-8d25-4b42-b780-11ff04ce1f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:29.121459 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f3326b-8d25-4b42-b780-11ff04ce1f42 admin admin] PATH_INFO: `/v3/projects/102645d12dbc4e3f8a91c56af17ee6e2` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:29.122590 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3326b-8d25-4b42-b780-11ff04ce1f42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=102645d12dbc4e3f8a91c56af17ee6e2)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:24:29.124474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f3326b-8d25-4b42-b780-11ff04ce1f42 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:24:29.140463 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:29.173158 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-17f3326b-8d25-4b42-b780-11ff04ce1f42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '102645d12dbc4e3f8a91c56af17ee6e2', 'request_id': 'req-17f3326b-8d25-4b42-b780-11ff04ce1f42'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:29.179317 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-17f3326b-8d25-4b42-b780-11ff04ce1f42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '102645d12dbc4e3f8a91c56af17ee6e2', 'request_id': 'req-17f3326b-8d25-4b42-b780-11ff04ce1f42'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:29.185078 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-17f3326b-8d25-4b42-b780-11ff04ce1f42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '102645d12dbc4e3f8a91c56af17ee6e2', 'request_id': 'req-17f3326b-8d25-4b42-b780-11ff04ce1f42'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:29.199072 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-17f3326b-8d25-4b42-b780-11ff04ce1f42 admin admin] The token cache is being invalidate because project 102645d12dbc4e3f8a91c56af17ee6e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:24:29.199369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-17f3326b-8d25-4b42-b780-11ff04ce1f42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:29.200527 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-17f3326b-8d25-4b42-b780-11ff04ce1f42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:24:29.202078 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1397/2780] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:24:29 2022] DELETE /identity/v3/projects/102645d12dbc4e3f8a91c56af17ee6e2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:24:38.768046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=70,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:39.196475 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=18,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:24:47.951651 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d869be-3b18-4e45-947b-94e59b87b445 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:24:47.952672 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d869be-3b18-4e45-947b-94e59b87b445 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:24:47.952796 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d869be-3b18-4e45-947b-94e59b87b445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:24:47.972625 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-b0d869be-3b18-4e45-947b-94e59b87b445 None None] MFA Rules not processed for user `347993dd122543a684681c1a0783962c`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:24:47.973872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b0d869be-3b18-4e45-947b-94e59b87b445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:24:48.028537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1384/2781] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:24:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:24:58.021832 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=25 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:16.711575 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3326b-8d25-4b42-b780-11ff04ce1f42 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:16.713338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-17f3326b-8d25-4b42-b780-11ff04ce1f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:16.788049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f3326b-8d25-4b42-b780-11ff04ce1f42 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:16.793536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b475610-3229-45b0-9973-968ba4bc2e50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:16.794168 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b475610-3229-45b0-9973-968ba4bc2e50 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:16.794448 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b475610-3229-45b0-9973-968ba4bc2e50 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:16.794721 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b475610-3229-45b0-9973-968ba4bc2e50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:16.796430 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2b475610-3229-45b0-9973-968ba4bc2e50 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:16.807051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b475610-3229-45b0-9973-968ba4bc2e50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:16.808606 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b475610-3229-45b0-9973-968ba4bc2e50 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:16.818023 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1398/2782] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:16 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:26.813368 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=34 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:34.145311 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d869be-3b18-4e45-947b-94e59b87b445 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:34.146680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-b0d869be-3b18-4e45-947b-94e59b87b445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:34.219983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d869be-3b18-4e45-947b-94e59b87b445 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:34.225823 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf783d28-9ea6-4887-9582-7ec83d7f655f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:34.226485 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf783d28-9ea6-4887-9582-7ec83d7f655f service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:34.226762 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf783d28-9ea6-4887-9582-7ec83d7f655f service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:34.227053 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf783d28-9ea6-4887-9582-7ec83d7f655f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:34.228763 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cf783d28-9ea6-4887-9582-7ec83d7f655f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:34.241684 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf783d28-9ea6-4887-9582-7ec83d7f655f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:34.243384 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf783d28-9ea6-4887-9582-7ec83d7f655f service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:34.253638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1385/2783] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:37.352147 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1f46c-6ae5-4cec-9dc2-b90c28cd3d06 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.352875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1f46c-6ae5-4cec-9dc2-b90c28cd3d06 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.352875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b1f46c-6ae5-4cec-9dc2-b90c28cd3d06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.372254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f3b1f46c-6ae5-4cec-9dc2-b90c28cd3d06 None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:37.373487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f3b1f46c-6ae5-4cec-9dc2-b90c28cd3d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:37.428433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1399/2784] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 19:26:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:37.474940 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf783d28-9ea6-4887-9582-7ec83d7f655f service placement] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:37.476414 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-cf783d28-9ea6-4887-9582-7ec83d7f655f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:37.545517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf783d28-9ea6-4887-9582-7ec83d7f655f service placement] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:37.550587 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a0348-a30f-4913-ab94-25f54c2c066b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:37.551184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a0348-a30f-4913-ab94-25f54c2c066b service swift] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:37.551459 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a0348-a30f-4913-ab94-25f54c2c066b service swift] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:37.551723 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7a0348-a30f-4913-ab94-25f54c2c066b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:37.560660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a0348-a30f-4913-ab94-25f54c2c066b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:37.562291 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7a0348-a30f-4913-ab94-25f54c2c066b service swift] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:37.570686 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1386/2785] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:26:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:26:47.421851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=25 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:47.566961 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=64 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:26:53.105851 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1f46c-6ae5-4cec-9dc2-b90c28cd3d06 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:53.107226 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f3b1f46c-6ae5-4cec-9dc2-b90c28cd3d06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:53.166237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b1f46c-6ae5-4cec-9dc2-b90c28cd3d06 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:53.171697 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cce968-2e4d-4f2d-83f7-5e5e593b53c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:53.172305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cce968-2e4d-4f2d-83f7-5e5e593b53c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:53.172580 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cce968-2e4d-4f2d-83f7-5e5e593b53c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:53.172846 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cce968-2e4d-4f2d-83f7-5e5e593b53c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:53.174570 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-70cce968-2e4d-4f2d-83f7-5e5e593b53c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:53.184795 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cce968-2e4d-4f2d-83f7-5e5e593b53c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:53.186306 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cce968-2e4d-4f2d-83f7-5e5e593b53c7 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:53.194704 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1400/2786] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:26:53.268023 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a0348-a30f-4913-ab94-25f54c2c066b service swift] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:53.269497 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7f7a0348-a30f-4913-ab94-25f54c2c066b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:53.281858 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7a0348-a30f-4913-ab94-25f54c2c066b service swift] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:53.285432 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699dbb2-ab5f-46ab-a6be-ad8eebb6c89b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:53.286017 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699dbb2-ab5f-46ab-a6be-ad8eebb6c89b service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:53.286294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699dbb2-ab5f-46ab-a6be-ad8eebb6c89b service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:53.286562 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6699dbb2-ab5f-46ab-a6be-ad8eebb6c89b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:53.288845 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-6699dbb2-ab5f-46ab-a6be-ad8eebb6c89b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:53.301790 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699dbb2-ab5f-46ab-a6be-ad8eebb6c89b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:53.303200 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6699dbb2-ab5f-46ab-a6be-ad8eebb6c89b service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:53.314017 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1387/2787] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:26:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:26:54.959213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476008d8-b7b8-4f48-9ded-7d6c2c34ed81 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:54.959666 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476008d8-b7b8-4f48-9ded-7d6c2c34ed81 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:54.959804 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476008d8-b7b8-4f48-9ded-7d6c2c34ed81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:54.978835 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-476008d8-b7b8-4f48-9ded-7d6c2c34ed81 None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:26:54.980028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-476008d8-b7b8-4f48-9ded-7d6c2c34ed81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:26:54.987171 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1401/2788] 10.222.0.65 () {58 vars in 1009 bytes} [Wed Aug 31 19:26:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:26:54.999283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699dbb2-ab5f-46ab-a6be-ad8eebb6c89b service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:26:55.011195 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6699dbb2-ab5f-46ab-a6be-ad8eebb6c89b service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:26:55.016145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52a2539-0d4f-442a-b3bc-82cde983b4cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:26:55.016730 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a2539-0d4f-442a-b3bc-82cde983b4cc service swift] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:26:55.017006 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a2539-0d4f-442a-b3bc-82cde983b4cc service swift] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:26:55.017355 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52a2539-0d4f-442a-b3bc-82cde983b4cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:26:55.025299 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a2539-0d4f-442a-b3bc-82cde983b4cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:26:55.026970 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52a2539-0d4f-442a-b3bc-82cde983b4cc service swift] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:26:55.034903 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1388/2789] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:26:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:27:04.973498 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=29 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:05.031742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=12 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:07.298075 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476008d8-b7b8-4f48-9ded-7d6c2c34ed81 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:07.307406 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476008d8-b7b8-4f48-9ded-7d6c2c34ed81 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:07.310917 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d2dc7-b984-47a9-ac04-e1cf1a66c176 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:07.311497 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d2dc7-b984-47a9-ac04-e1cf1a66c176 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:07.311777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d2dc7-b984-47a9-ac04-e1cf1a66c176 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:07.312049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d2dc7-b984-47a9-ac04-e1cf1a66c176 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:07.313762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ff5d2dc7-b984-47a9-ac04-e1cf1a66c176 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:07.326806 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d2dc7-b984-47a9-ac04-e1cf1a66c176 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:07.328219 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d2dc7-b984-47a9-ac04-e1cf1a66c176 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:07.339594 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1402/2790] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:27:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:27:14.795555 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7524d3-84c2-4a02-9f52-fec63aae2a28 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:14.796081 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7524d3-84c2-4a02-9f52-fec63aae2a28 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:14.796187 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7524d3-84c2-4a02-9f52-fec63aae2a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:14.814889 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ac7524d3-84c2-4a02-9f52-fec63aae2a28 None None] MFA Rules not processed for user `dd730375a996433d982b26c8339ec4ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:27:14.816063 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ac7524d3-84c2-4a02-9f52-fec63aae2a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:27:14.823106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1389/2791] 10.222.0.65 () {58 vars in 967 bytes} [Wed Aug 31 19:27:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:27:14.835409 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d2dc7-b984-47a9-ac04-e1cf1a66c176 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:27:14.847482 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d2dc7-b984-47a9-ac04-e1cf1a66c176 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:27:14.853159 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7aa902-080e-4d8c-a7a6-1a2a9d8e5521 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '586715d11b924e2cb6f44030efd77619', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:27:14.853776 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7aa902-080e-4d8c-a7a6-1a2a9d8e5521 service swift] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:27:14.854054 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7aa902-080e-4d8c-a7a6-1a2a9d8e5521 service swift] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:27:14.854326 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7aa902-080e-4d8c-a7a6-1a2a9d8e5521 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:27:14.862309 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7aa902-080e-4d8c-a7a6-1a2a9d8e5521 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:27:14.863864 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7aa902-080e-4d8c-a7a6-1a2a9d8e5521 service swift] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:27:14.871931 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1403/2792] 10.222.0.65 () {60 vars in 1384 bytes} [Wed Aug 31 19:27:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 31 19:27:24.809689 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:27:24.868727 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=12 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:11.671111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7524d3-84c2-4a02-9f52-fec63aae2a28 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:11.680376 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7524d3-84c2-4a02-9f52-fec63aae2a28 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:11.684010 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee96a4-0053-41b2-ad31-a704003b3b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:11.684596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee96a4-0053-41b2-ad31-a704003b3b9d service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:11.684872 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee96a4-0053-41b2-ad31-a704003b3b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:11.685140 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dee96a4-0053-41b2-ad31-a704003b3b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:11.686864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8dee96a4-0053-41b2-ad31-a704003b3b9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:11.700356 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee96a4-0053-41b2-ad31-a704003b3b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:11.701805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dee96a4-0053-41b2-ad31-a704003b3b9d service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:11.712525 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1390/2793] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:19.361201 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7aa902-080e-4d8c-a7a6-1a2a9d8e5521 service swift] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:19.370522 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7aa902-080e-4d8c-a7a6-1a2a9d8e5521 service swift] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:19.374638 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011c4e4-e7d6-4bf2-a644-09e9dff64563 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:19.375218 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011c4e4-e7d6-4bf2-a644-09e9dff64563 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:19.375495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011c4e4-e7d6-4bf2-a644-09e9dff64563 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:19.375762 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7011c4e4-e7d6-4bf2-a644-09e9dff64563 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:19.377482 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7011c4e4-e7d6-4bf2-a644-09e9dff64563 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:19.438985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011c4e4-e7d6-4bf2-a644-09e9dff64563 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:19.440397 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7011c4e4-e7d6-4bf2-a644-09e9dff64563 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:19.449836 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1404/2794] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:21.707748 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=6 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:22.465408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee96a4-0053-41b2-ad31-a704003b3b9d service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:22.467449 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-8dee96a4-0053-41b2-ad31-a704003b3b9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:22.484101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dee96a4-0053-41b2-ad31-a704003b3b9d service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:22.489007 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1305f8-51c2-4b97-bb36-7042fea76bd8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:22.489652 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1305f8-51c2-4b97-bb36-7042fea76bd8 service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:22.489930 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1305f8-51c2-4b97-bb36-7042fea76bd8 service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:22.490196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1305f8-51c2-4b97-bb36-7042fea76bd8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:22.491999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4e1305f8-51c2-4b97-bb36-7042fea76bd8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:22.504407 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1305f8-51c2-4b97-bb36-7042fea76bd8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:22.506047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1305f8-51c2-4b97-bb36-7042fea76bd8 service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:22.524651 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1391/2795] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:22.545151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011c4e4-e7d6-4bf2-a644-09e9dff64563 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:22.547449 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7011c4e4-e7d6-4bf2-a644-09e9dff64563 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:22.564292 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7011c4e4-e7d6-4bf2-a644-09e9dff64563 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:22.569369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fc0513-3053-4334-8029-f3548fb9786c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:22.570136 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc0513-3053-4334-8029-f3548fb9786c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:22.570413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc0513-3053-4334-8029-f3548fb9786c admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:22.570683 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc0513-3053-4334-8029-f3548fb9786c admin admin] PATH_INFO: `/v3/users/347993dd122543a684681c1a0783962c` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:22.571781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc0513-3053-4334-8029-f3548fb9786c admin admin] RBAC: Authorizing `identity:delete_user(user_id=347993dd122543a684681c1a0783962c)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:22.573733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fc0513-3053-4334-8029-f3548fb9786c admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:22.620674 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-58fc0513-3053-4334-8029-f3548fb9786c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '347993dd122543a684681c1a0783962c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:22.627718 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-58fc0513-3053-4334-8029-f3548fb9786c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '347993dd122543a684681c1a0783962c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:22.635424 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-58fc0513-3053-4334-8029-f3548fb9786c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '347993dd122543a684681c1a0783962c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:22.636074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-58fc0513-3053-4334-8029-f3548fb9786c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '347993dd122543a684681c1a0783962c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:22.648438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-58fc0513-3053-4334-8029-f3548fb9786c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '347993dd122543a684681c1a0783962c'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:22.654748 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1405/2796] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:28:22 2022] DELETE /identity/v3/users/347993dd122543a684681c1a0783962c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:22.822329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1305f8-51c2-4b97-bb36-7042fea76bd8 service placement] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:22.833712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1305f8-51c2-4b97-bb36-7042fea76bd8 service placement] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:22.894149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b725198-a495-4603-b2df-3ec74b2e2c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:22.894768 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b725198-a495-4603-b2df-3ec74b2e2c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:22.895042 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b725198-a495-4603-b2df-3ec74b2e2c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:22.895309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b725198-a495-4603-b2df-3ec74b2e2c3e admin admin] PATH_INFO: `/v3/projects/12835bbc76594286925e297e824e0d91` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:22.896407 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b725198-a495-4603-b2df-3ec74b2e2c3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=12835bbc76594286925e297e824e0d91)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:22.898324 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b725198-a495-4603-b2df-3ec74b2e2c3e admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:22.943680 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b725198-a495-4603-b2df-3ec74b2e2c3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12835bbc76594286925e297e824e0d91', 'request_id': 'req-0b725198-a495-4603-b2df-3ec74b2e2c3e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:22.950636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b725198-a495-4603-b2df-3ec74b2e2c3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12835bbc76594286925e297e824e0d91', 'request_id': 'req-0b725198-a495-4603-b2df-3ec74b2e2c3e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:22.956127 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b725198-a495-4603-b2df-3ec74b2e2c3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12835bbc76594286925e297e824e0d91', 'request_id': 'req-0b725198-a495-4603-b2df-3ec74b2e2c3e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:22.967393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b725198-a495-4603-b2df-3ec74b2e2c3e admin admin] The token cache is being invalidate because project 12835bbc76594286925e297e824e0d91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:28:22.967676 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b725198-a495-4603-b2df-3ec74b2e2c3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:22.968550 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-0b725198-a495-4603-b2df-3ec74b2e2c3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:28:22.970045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1392/2797] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:28:22 2022] DELETE /identity/v3/projects/12835bbc76594286925e297e824e0d91 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:22.975155 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034c552-45b9-425b-805c-29f5731fced9 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:22.975468 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034c552-45b9-425b-805c-29f5731fced9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:22.975742 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3034c552-45b9-425b-805c-29f5731fced9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:22.996238 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-3034c552-45b9-425b-805c-29f5731fced9 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:22.998039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3034c552-45b9-425b-805c-29f5731fced9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:23.059452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1406/2798] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:23.067789 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015c127-1a4e-4dd1-9366-800e5b7cda15 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:23.068072 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015c127-1a4e-4dd1-9366-800e5b7cda15 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:23.068346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4015c127-1a4e-4dd1-9366-800e5b7cda15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:23.088181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4015c127-1a4e-4dd1-9366-800e5b7cda15 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:23.089408 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4015c127-1a4e-4dd1-9366-800e5b7cda15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:23.097538 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1393/2799] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:23.101855 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034c552-45b9-425b-805c-29f5731fced9 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:23.112660 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3034c552-45b9-425b-805c-29f5731fced9 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:23.117337 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac6f5b-e93c-4cc6-93e0-ee46d3f87122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:23.117911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac6f5b-e93c-4cc6-93e0-ee46d3f87122 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:23.118188 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac6f5b-e93c-4cc6-93e0-ee46d3f87122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:23.118452 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ac6f5b-e93c-4cc6-93e0-ee46d3f87122 admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:23.119150 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac6f5b-e93c-4cc6-93e0-ee46d3f87122 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:28:23.119365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac6f5b-e93c-4cc6-93e0-ee46d3f87122 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:23.120949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ac6f5b-e93c-4cc6-93e0-ee46d3f87122 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:23.123100 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:23.135857 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1407/2800] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:28:23.140403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015c127-1a4e-4dd1-9366-800e5b7cda15 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:23.149398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4015c127-1a4e-4dd1-9366-800e5b7cda15 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:23.153959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd1d3c-6729-45bd-952e-1a80e02de099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:23.154536 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd1d3c-6729-45bd-952e-1a80e02de099 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:23.154806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd1d3c-6729-45bd-952e-1a80e02de099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:23.155082 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bd1d3c-6729-45bd-952e-1a80e02de099 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:23.155685 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd1d3c-6729-45bd-952e-1a80e02de099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:23.157579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bd1d3c-6729-45bd-952e-1a80e02de099 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:23.183682 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1394/2801] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:28:23 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:28:23.188304 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac6f5b-e93c-4cc6-93e0-ee46d3f87122 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:23.198227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ac6f5b-e93c-4cc6-93e0-ee46d3f87122 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:23.254635 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407ffe8-504a-4ab7-ab88-e90fc3108cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:23.255237 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407ffe8-504a-4ab7-ab88-e90fc3108cfa admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:23.255515 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407ffe8-504a-4ab7-ab88-e90fc3108cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:23.255779 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f407ffe8-504a-4ab7-ab88-e90fc3108cfa admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:23.256386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407ffe8-504a-4ab7-ab88-e90fc3108cfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:23.258311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f407ffe8-504a-4ab7-ab88-e90fc3108cfa admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:23.280672 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1408/2802] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:28:23 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:28:23.284814 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd1d3c-6729-45bd-952e-1a80e02de099 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:23.295154 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bd1d3c-6729-45bd-952e-1a80e02de099 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:23.299805 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98550d-c72a-40d2-baaa-4f76a40b1e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:23.300418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98550d-c72a-40d2-baaa-4f76a40b1e28 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:23.300695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98550d-c72a-40d2-baaa-4f76a40b1e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:23.300964 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac98550d-c72a-40d2-baaa-4f76a40b1e28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:23.301640 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98550d-c72a-40d2-baaa-4f76a40b1e28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:23.303413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac98550d-c72a-40d2-baaa-4f76a40b1e28 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:23.313114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1395/2803] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:23.316875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407ffe8-504a-4ab7-ab88-e90fc3108cfa admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:23.327102 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f407ffe8-504a-4ab7-ab88-e90fc3108cfa admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:23.331612 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11ede7d-a6dd-48fc-80b4-87eebb7e66ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:23.332173 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ede7d-a6dd-48fc-80b4-87eebb7e66ae admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:23.332520 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ede7d-a6dd-48fc-80b4-87eebb7e66ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:23.332792 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11ede7d-a6dd-48fc-80b4-87eebb7e66ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:23.333448 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ede7d-a6dd-48fc-80b4-87eebb7e66ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:23.335034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11ede7d-a6dd-48fc-80b4-87eebb7e66ae admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:23.346094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1409/2804] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:28:23.350046 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98550d-c72a-40d2-baaa-4f76a40b1e28 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:23.359838 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac98550d-c72a-40d2-baaa-4f76a40b1e28 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:23.364399 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c08ee-dbc8-4eed-b0c6-992225fac7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:23.365031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c08ee-dbc8-4eed-b0c6-992225fac7ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:23.365342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c08ee-dbc8-4eed-b0c6-992225fac7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:23.365615 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0c08ee-dbc8-4eed-b0c6-992225fac7ac admin admin] PATH_INFO: `/v3/projects/548f21e0f1dc4c2da5e743e6d01154a3/users/745deef0124c480cb48ebe7efaea546a/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:23.379715 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c08ee-dbc8-4eed-b0c6-992225fac7ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=548f21e0f1dc4c2da5e743e6d01154a3, user_id=745deef0124c480cb48ebe7efaea546a, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:23.382592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0c08ee-dbc8-4eed-b0c6-992225fac7ac admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:23.395842 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1396/2805] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:28:23 2022] PUT /identity/v3/projects/548f21e0f1dc4c2da5e743e6d01154a3/users/745deef0124c480cb48ebe7efaea546a/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:28:23.400905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9543f8-0046-4644-adad-8205d60e187c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:23.401211 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9543f8-0046-4644-adad-8205d60e187c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:23.401478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9543f8-0046-4644-adad-8205d60e187c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:23.420373 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-bc9543f8-0046-4644-adad-8205d60e187c None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:23.421581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-bc9543f8-0046-4644-adad-8205d60e187c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:23.479351 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1410/2806] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:23.488905 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c08ee-dbc8-4eed-b0c6-992225fac7ac admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:23.490623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ab0c08ee-dbc8-4eed-b0c6-992225fac7ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:23.505229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:23.548676 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0c08ee-dbc8-4eed-b0c6-992225fac7ac admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:23.552246 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4332f2-480a-4a88-b318-a03479de3e83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:23.552835 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4332f2-480a-4a88-b318-a03479de3e83 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:23.553111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4332f2-480a-4a88-b318-a03479de3e83 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:23.553406 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4332f2-480a-4a88-b318-a03479de3e83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:23.562639 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4332f2-480a-4a88-b318-a03479de3e83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:23.564050 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4332f2-480a-4a88-b318-a03479de3e83 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:23.573233 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1397/2807] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:27.251921 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6329f-8bc4-48f3-8233-804abd8d7c0c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:27.252415 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6329f-8bc4-48f3-8233-804abd8d7c0c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:27.253009 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6329f-8bc4-48f3-8233-804abd8d7c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:27.282067 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8ea6329f-8bc4-48f3-8233-804abd8d7c0c None None] MFA Rules not processed for user `745deef0124c480cb48ebe7efaea546a`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:27.283346 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8ea6329f-8bc4-48f3-8233-804abd8d7c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:27.369642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1411/2808] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:28:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:27.372617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4332f2-480a-4a88-b318-a03479de3e83 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:27.374077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-fc4332f2-480a-4a88-b318-a03479de3e83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:27.449841 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4332f2-480a-4a88-b318-a03479de3e83 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:27.455417 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64563296-f0d0-4210-b798-cfd2350de7fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:27.456019 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64563296-f0d0-4210-b798-cfd2350de7fb service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:27.456296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64563296-f0d0-4210-b798-cfd2350de7fb service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:27.456571 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64563296-f0d0-4210-b798-cfd2350de7fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:27.464596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64563296-f0d0-4210-b798-cfd2350de7fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:27.466770 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64563296-f0d0-4210-b798-cfd2350de7fb service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:27.477226 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1398/2809] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:28.821621 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33008785-1315-407a-86c6-a51b0406c0ff None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:28.822127 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33008785-1315-407a-86c6-a51b0406c0ff None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:28.822258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33008785-1315-407a-86c6-a51b0406c0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:28.829618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:28.842790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-33008785-1315-407a-86c6-a51b0406c0ff None None] MFA Rules not processed for user `745deef0124c480cb48ebe7efaea546a`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:28:28.844226 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-33008785-1315-407a-86c6-a51b0406c0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:28.851120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1412/2810] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:28:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:28:29.177127 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64563296-f0d0-4210-b798-cfd2350de7fb service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:29.187389 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64563296-f0d0-4210-b798-cfd2350de7fb service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:29.191201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cd095-66e7-49c6-8521-c3a38175940a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:29.191934 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cd095-66e7-49c6-8521-c3a38175940a service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:29.192327 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cd095-66e7-49c6-8521-c3a38175940a service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:29.192666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cd095-66e7-49c6-8521-c3a38175940a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:29.194726 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-998cd095-66e7-49c6-8521-c3a38175940a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:29.204896 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cd095-66e7-49c6-8521-c3a38175940a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:29.206356 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cd095-66e7-49c6-8521-c3a38175940a service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:29.218843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1399/2811] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:29.580160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33008785-1315-407a-86c6-a51b0406c0ff None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:29.582218 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-33008785-1315-407a-86c6-a51b0406c0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:29.659537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33008785-1315-407a-86c6-a51b0406c0ff None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:29.664773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c77bed-aa40-4827-a9f6-948a5efe488f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:29.665600 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c77bed-aa40-4827-a9f6-948a5efe488f service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:29.665971 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c77bed-aa40-4827-a9f6-948a5efe488f service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:29.666314 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c77bed-aa40-4827-a9f6-948a5efe488f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:29.668161 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-29c77bed-aa40-4827-a9f6-948a5efe488f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:29.680928 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c77bed-aa40-4827-a9f6-948a5efe488f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:29.682573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c77bed-aa40-4827-a9f6-948a5efe488f service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:29.694863 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1413/2812] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:31.628910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cd095-66e7-49c6-8521-c3a38175940a service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:31.630525 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-998cd095-66e7-49c6-8521-c3a38175940a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:31.641251 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cd095-66e7-49c6-8521-c3a38175940a service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:31.645075 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b1ea7-d2e7-4cfa-8a8f-4cd62c0eabce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:31.645720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b1ea7-d2e7-4cfa-8a8f-4cd62c0eabce service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:31.645997 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b1ea7-d2e7-4cfa-8a8f-4cd62c0eabce service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:31.646326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874b1ea7-d2e7-4cfa-8a8f-4cd62c0eabce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:31.648182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-874b1ea7-d2e7-4cfa-8a8f-4cd62c0eabce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:31.659163 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b1ea7-d2e7-4cfa-8a8f-4cd62c0eabce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:31.660596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874b1ea7-d2e7-4cfa-8a8f-4cd62c0eabce service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:31.669707 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1400/2813] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:39.689657 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=34 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:41.665293 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=66 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:28:42.724044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c77bed-aa40-4827-a9f6-948a5efe488f service placement] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:28:42.736304 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c77bed-aa40-4827-a9f6-948a5efe488f service placement] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:28:42.741716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06073d5d-4afe-4652-9cd8-03da01590d92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:28:42.742340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06073d5d-4afe-4652-9cd8-03da01590d92 service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:28:42.742618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06073d5d-4afe-4652-9cd8-03da01590d92 service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:28:42.742903 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06073d5d-4afe-4652-9cd8-03da01590d92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:28:42.744620 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-06073d5d-4afe-4652-9cd8-03da01590d92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:28:42.756897 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06073d5d-4afe-4652-9cd8-03da01590d92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:28:42.758510 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06073d5d-4afe-4652-9cd8-03da01590d92 service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:28:42.770082 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1414/2814] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:28:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:28:52.765391 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=6 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:31:43.160779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b1ea7-d2e7-4cfa-8a8f-4cd62c0eabce service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:31:43.162222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-874b1ea7-d2e7-4cfa-8a8f-4cd62c0eabce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:31:43.248440 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874b1ea7-d2e7-4cfa-8a8f-4cd62c0eabce service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:31:43.253500 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef7a4c-722e-473f-b883-7f30366b3de9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:31:43.254100 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef7a4c-722e-473f-b883-7f30366b3de9 service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:31:43.254372 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef7a4c-722e-473f-b883-7f30366b3de9 service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:31:43.254637 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edef7a4c-722e-473f-b883-7f30366b3de9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:31:43.256332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-edef7a4c-722e-473f-b883-7f30366b3de9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:31:43.268502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef7a4c-722e-473f-b883-7f30366b3de9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:31:43.270086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edef7a4c-722e-473f-b883-7f30366b3de9 service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:31:43.280397 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1401/2815] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:31:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:31:53.276281 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=37 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:32:54.643609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06073d5d-4afe-4652-9cd8-03da01590d92 service nova] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:32:54.679905 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06073d5d-4afe-4652-9cd8-03da01590d92 service nova] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:32:54.683484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e6554-b71f-4c34-bbc5-1f051aa0b93a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:32:54.684073 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e6554-b71f-4c34-bbc5-1f051aa0b93a service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:54.684351 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e6554-b71f-4c34-bbc5-1f051aa0b93a service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:54.684619 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e6554-b71f-4c34-bbc5-1f051aa0b93a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:54.686846 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-944e6554-b71f-4c34-bbc5-1f051aa0b93a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:32:54.719428 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e6554-b71f-4c34-bbc5-1f051aa0b93a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:32:54.720846 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e6554-b71f-4c34-bbc5-1f051aa0b93a service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:32:54.740918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1415/2816] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:32:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:32:56.721102 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef7a4c-722e-473f-b883-7f30366b3de9 service placement] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:32:56.734753 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edef7a4c-722e-473f-b883-7f30366b3de9 service placement] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:32:56.740782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85908fa-6479-43d6-9044-3bb3e7e7734e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:32:56.741435 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85908fa-6479-43d6-9044-3bb3e7e7734e service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:32:56.741714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85908fa-6479-43d6-9044-3bb3e7e7734e service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:32:56.741983 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85908fa-6479-43d6-9044-3bb3e7e7734e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:32:56.744222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f85908fa-6479-43d6-9044-3bb3e7e7734e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:32:56.759894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85908fa-6479-43d6-9044-3bb3e7e7734e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:32:56.761573 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85908fa-6479-43d6-9044-3bb3e7e7734e service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:32:56.773914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1402/2817] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:32:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:33:00.085913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69791a0-f4c2-484c-ba00-33963862f7e4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:00.086515 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69791a0-f4c2-484c-ba00-33963862f7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:00.086515 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69791a0-f4c2-484c-ba00-33963862f7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:00.106026 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-f69791a0-f4c2-484c-ba00-33963862f7e4 None None] MFA Rules not processed for user `745deef0124c480cb48ebe7efaea546a`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:33:00.107234 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f69791a0-f4c2-484c-ba00-33963862f7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:00.115957 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1416/2818] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:33:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:33:06.768424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=6 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:33:08.097048 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85908fa-6479-43d6-9044-3bb3e7e7734e service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:08.108972 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85908fa-6479-43d6-9044-3bb3e7e7734e service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:08.115394 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063b4d6-ea95-4610-b890-60dc13008441 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:08.115996 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063b4d6-ea95-4610-b890-60dc13008441 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:08.116273 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063b4d6-ea95-4610-b890-60dc13008441 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:08.116539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3063b4d6-ea95-4610-b890-60dc13008441 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:08.118317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3063b4d6-ea95-4610-b890-60dc13008441 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:33:08.133145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3063b4d6-ea95-4610-b890-60dc13008441 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:08.134727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3063b4d6-ea95-4610-b890-60dc13008441 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:08.145740 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1403/2819] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:33:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:33:10.099985 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=33 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:33:18.141637 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=6 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:33:37.851658 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69791a0-f4c2-484c-ba00-33963862f7e4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:33:37.861564 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69791a0-f4c2-484c-ba00-33963862f7e4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:33:37.865821 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677e273-ef0a-4273-b1d3-9d16a899b763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:33:37.866408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677e273-ef0a-4273-b1d3-9d16a899b763 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:33:37.866681 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677e273-ef0a-4273-b1d3-9d16a899b763 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:33:37.866946 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f677e273-ef0a-4273-b1d3-9d16a899b763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:33:37.874052 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677e273-ef0a-4273-b1d3-9d16a899b763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:33:37.875454 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f677e273-ef0a-4273-b1d3-9d16a899b763 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:33:37.883305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1417/2820] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:33:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:33:47.879814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=6 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:34:26.397377 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063b4d6-ea95-4610-b890-60dc13008441 service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:34:26.398868 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-3063b4d6-ea95-4610-b890-60dc13008441 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:34:26.414959 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3063b4d6-ea95-4610-b890-60dc13008441 service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:34:26.420457 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aec743-1a30-46a3-9d38-88cee26e9d17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:34:26.421065 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aec743-1a30-46a3-9d38-88cee26e9d17 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:34:26.421462 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aec743-1a30-46a3-9d38-88cee26e9d17 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:34:26.421738 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aec743-1a30-46a3-9d38-88cee26e9d17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:34:26.430018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aec743-1a30-46a3-9d38-88cee26e9d17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:34:26.431592 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aec743-1a30-46a3-9d38-88cee26e9d17 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:34:26.440221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1404/2821] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:34:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:34:36.436322 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=6 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:34:37.061562 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677e273-ef0a-4273-b1d3-9d16a899b763 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:34:37.062949 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f677e273-ef0a-4273-b1d3-9d16a899b763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:34:37.139175 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f677e273-ef0a-4273-b1d3-9d16a899b763 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:34:37.142756 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e334a15-efe8-42aa-a233-355ddbbc8761 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:34:37.143347 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e334a15-efe8-42aa-a233-355ddbbc8761 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:34:37.143626 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e334a15-efe8-42aa-a233-355ddbbc8761 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:34:37.143902 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e334a15-efe8-42aa-a233-355ddbbc8761 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:34:37.145622 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2e334a15-efe8-42aa-a233-355ddbbc8761 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:34:37.159238 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e334a15-efe8-42aa-a233-355ddbbc8761 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:34:37.160679 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e334a15-efe8-42aa-a233-355ddbbc8761 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:34:37.172028 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1418/2822] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:34:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:34:47.167257 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=35 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:34:57.070277 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aec743-1a30-46a3-9d38-88cee26e9d17 service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:34:57.079666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aec743-1a30-46a3-9d38-88cee26e9d17 service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:34:57.083270 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bc732-765f-4470-a39a-f55613f6860d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:34:57.083854 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bc732-765f-4470-a39a-f55613f6860d service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:34:57.084132 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bc732-765f-4470-a39a-f55613f6860d service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:34:57.084968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0bc732-765f-4470-a39a-f55613f6860d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:34:57.094227 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0bc732-765f-4470-a39a-f55613f6860d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:34:57.095631 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0bc732-765f-4470-a39a-f55613f6860d service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:34:57.122451 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1405/2823] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:34:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:35:00.177848 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e334a15-efe8-42aa-a233-355ddbbc8761 service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:00.195775 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e334a15-efe8-42aa-a233-355ddbbc8761 service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:00.204854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cc6ad-8fa6-4b92-86f9-77c0143192ab service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:00.205802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cc6ad-8fa6-4b92-86f9-77c0143192ab service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:00.206340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cc6ad-8fa6-4b92-86f9-77c0143192ab service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:00.206707 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36cc6ad-8fa6-4b92-86f9-77c0143192ab service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:00.208579 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f36cc6ad-8fa6-4b92-86f9-77c0143192ab service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:00.225448 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36cc6ad-8fa6-4b92-86f9-77c0143192ab service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:00.227198 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36cc6ad-8fa6-4b92-86f9-77c0143192ab service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:00.238943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1419/2824] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:35:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:35:00.241106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bc732-765f-4470-a39a-f55613f6860d service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:00.252749 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0bc732-765f-4470-a39a-f55613f6860d service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:00.257704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed786c1c-26ae-4e9a-b8bd-ba1a7232eab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:00.258365 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed786c1c-26ae-4e9a-b8bd-ba1a7232eab1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:00.258860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed786c1c-26ae-4e9a-b8bd-ba1a7232eab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:00.258860 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed786c1c-26ae-4e9a-b8bd-ba1a7232eab1 admin admin] PATH_INFO: `/v3/users/745deef0124c480cb48ebe7efaea546a` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:00.260141 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed786c1c-26ae-4e9a-b8bd-ba1a7232eab1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=745deef0124c480cb48ebe7efaea546a)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:00.263793 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed786c1c-26ae-4e9a-b8bd-ba1a7232eab1 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:00.320933 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ed786c1c-26ae-4e9a-b8bd-ba1a7232eab1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '745deef0124c480cb48ebe7efaea546a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:00.327783 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ed786c1c-26ae-4e9a-b8bd-ba1a7232eab1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '745deef0124c480cb48ebe7efaea546a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:00.340201 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ed786c1c-26ae-4e9a-b8bd-ba1a7232eab1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '745deef0124c480cb48ebe7efaea546a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:00.340871 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ed786c1c-26ae-4e9a-b8bd-ba1a7232eab1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '745deef0124c480cb48ebe7efaea546a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:00.341491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-ed786c1c-26ae-4e9a-b8bd-ba1a7232eab1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '745deef0124c480cb48ebe7efaea546a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:00.354400 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1406/2825] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:35:00 2022] DELETE /identity/v3/users/745deef0124c480cb48ebe7efaea546a => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:35:00.529332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cc6ad-8fa6-4b92-86f9-77c0143192ab service placement] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:00.541934 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36cc6ad-8fa6-4b92-86f9-77c0143192ab service placement] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:00.601726 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48088a-c233-4adc-8a58-4dd98181a45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:00.602348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48088a-c233-4adc-8a58-4dd98181a45f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:00.602626 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48088a-c233-4adc-8a58-4dd98181a45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:00.602902 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48088a-c233-4adc-8a58-4dd98181a45f admin admin] PATH_INFO: `/v3/projects/548f21e0f1dc4c2da5e743e6d01154a3` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:00.604012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48088a-c233-4adc-8a58-4dd98181a45f admin admin] RBAC: Authorizing `identity:delete_project(project_id=548f21e0f1dc4c2da5e743e6d01154a3)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:00.605930 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d48088a-c233-4adc-8a58-4dd98181a45f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:00.653234 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1d48088a-c233-4adc-8a58-4dd98181a45f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '548f21e0f1dc4c2da5e743e6d01154a3', 'request_id': 'req-1d48088a-c233-4adc-8a58-4dd98181a45f'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:00.659120 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1d48088a-c233-4adc-8a58-4dd98181a45f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '548f21e0f1dc4c2da5e743e6d01154a3', 'request_id': 'req-1d48088a-c233-4adc-8a58-4dd98181a45f'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:00.664843 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1d48088a-c233-4adc-8a58-4dd98181a45f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '548f21e0f1dc4c2da5e743e6d01154a3', 'request_id': 'req-1d48088a-c233-4adc-8a58-4dd98181a45f'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:00.688281 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1d48088a-c233-4adc-8a58-4dd98181a45f admin admin] The token cache is being invalidate because project 548f21e0f1dc4c2da5e743e6d01154a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:35:00.688562 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1d48088a-c233-4adc-8a58-4dd98181a45f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:00.689170 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-1d48088a-c233-4adc-8a58-4dd98181a45f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:00.690643 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1420/2826] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:35:00 2022] DELETE /identity/v3/projects/548f21e0f1dc4c2da5e743e6d01154a3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:00.696228 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfd3f2-386c-43e0-b6b1-6facb0552665 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:00.696509 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfd3f2-386c-43e0-b6b1-6facb0552665 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:00.696779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfd3f2-386c-43e0-b6b1-6facb0552665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:00.753613 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-56bfd3f2-386c-43e0-b6b1-6facb0552665 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:00.754821 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-56bfd3f2-386c-43e0-b6b1-6facb0552665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:00.812514 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1407/2827] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:00.820757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0f4bc-90ae-4335-a3f3-348875bcac11 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:00.821039 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0f4bc-90ae-4335-a3f3-348875bcac11 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:00.821348 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d0f4bc-90ae-4335-a3f3-348875bcac11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:00.841209 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-c5d0f4bc-90ae-4335-a3f3-348875bcac11 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:00.842834 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-c5d0f4bc-90ae-4335-a3f3-348875bcac11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:00.850938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1421/2828] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:00.855038 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfd3f2-386c-43e0-b6b1-6facb0552665 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:00.864613 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfd3f2-386c-43e0-b6b1-6facb0552665 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:00.869110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e613f36-209b-4166-8512-84c750ced2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:00.869727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e613f36-209b-4166-8512-84c750ced2a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:00.870003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e613f36-209b-4166-8512-84c750ced2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:00.870274 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e613f36-209b-4166-8512-84c750ced2a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:00.870911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e613f36-209b-4166-8512-84c750ced2a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:35:00.871205 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e613f36-209b-4166-8512-84c750ced2a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:00.872779 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e613f36-209b-4166-8512-84c750ced2a6 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:00.881671 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:00.885146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1408/2829] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:35:00.889367 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0f4bc-90ae-4335-a3f3-348875bcac11 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:00.899209 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d0f4bc-90ae-4335-a3f3-348875bcac11 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:00.903861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc4b60-df79-4bd5-8cd6-57ee1c10e541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:00.904445 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc4b60-df79-4bd5-8cd6-57ee1c10e541 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:00.904726 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc4b60-df79-4bd5-8cd6-57ee1c10e541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:00.904999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc4b60-df79-4bd5-8cd6-57ee1c10e541 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:00.905654 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc4b60-df79-4bd5-8cd6-57ee1c10e541 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:00.907532 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fc4b60-df79-4bd5-8cd6-57ee1c10e541 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:00.943128 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1422/2830] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:35:00 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:35:00.948976 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e613f36-209b-4166-8512-84c750ced2a6 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:00.958791 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e613f36-209b-4166-8512-84c750ced2a6 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:01.013501 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f1a49-0ae4-4dd1-9287-e44867891c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:01.014088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1a49-0ae4-4dd1-9287-e44867891c9f admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:01.014368 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1a49-0ae4-4dd1-9287-e44867891c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:01.014636 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51f1a49-0ae4-4dd1-9287-e44867891c9f admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:01.015262 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f1a49-0ae4-4dd1-9287-e44867891c9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:01.017155 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51f1a49-0ae4-4dd1-9287-e44867891c9f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:01.038800 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1409/2831] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:35:00 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:35:01.042948 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc4b60-df79-4bd5-8cd6-57ee1c10e541 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:01.053885 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fc4b60-df79-4bd5-8cd6-57ee1c10e541 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:01.058650 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdeee2-b232-4c19-858b-e023ff798b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:01.059240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdeee2-b232-4c19-858b-e023ff798b1a admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:01.059523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdeee2-b232-4c19-858b-e023ff798b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:01.059790 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cdeee2-b232-4c19-858b-e023ff798b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:01.060441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cdeee2-b232-4c19-858b-e023ff798b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:01.062118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cdeee2-b232-4c19-858b-e023ff798b1a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:01.075408 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1423/2832] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:35:01.076413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f1a49-0ae4-4dd1-9287-e44867891c9f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:01.086044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51f1a49-0ae4-4dd1-9287-e44867891c9f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:01.091778 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1c3884-e945-486d-a284-5cb287a437be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:01.092354 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c3884-e945-486d-a284-5cb287a437be admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:01.092623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c3884-e945-486d-a284-5cb287a437be admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:01.092883 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1c3884-e945-486d-a284-5cb287a437be admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:01.093536 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1c3884-e945-486d-a284-5cb287a437be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:01.095095 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1c3884-e945-486d-a284-5cb287a437be admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:01.105342 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1410/2833] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:35:01.109328 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdeee2-b232-4c19-858b-e023ff798b1a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:01.120344 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cdeee2-b232-4c19-858b-e023ff798b1a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:01.125014 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73974f3f-c34f-4bf1-ade9-391c364a5a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:01.125697 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73974f3f-c34f-4bf1-ade9-391c364a5a2f admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:01.125978 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73974f3f-c34f-4bf1-ade9-391c364a5a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:01.126250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73974f3f-c34f-4bf1-ade9-391c364a5a2f admin admin] PATH_INFO: `/v3/projects/0a7314d447be4a3cb0bbcb6b82be76a3/users/464229f012424659878cf01ee92145e0/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:01.141422 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73974f3f-c34f-4bf1-ade9-391c364a5a2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a7314d447be4a3cb0bbcb6b82be76a3, user_id=464229f012424659878cf01ee92145e0, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:01.144049 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73974f3f-c34f-4bf1-ade9-391c364a5a2f admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:01.146254 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:01.158265 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1424/2834] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:35:01 2022] PUT /identity/v3/projects/0a7314d447be4a3cb0bbcb6b82be76a3/users/464229f012424659878cf01ee92145e0/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:01.168341 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e4067-1101-45f2-b74d-064ce0bd557d None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:01.168622 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e4067-1101-45f2-b74d-064ce0bd557d None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:01.168891 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e4067-1101-45f2-b74d-064ce0bd557d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:01.198087 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-355e4067-1101-45f2-b74d-064ce0bd557d None None] MFA Rules not processed for user `464229f012424659878cf01ee92145e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:01.199329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-355e4067-1101-45f2-b74d-064ce0bd557d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:01.269928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1411/2835] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:01.281221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73974f3f-c34f-4bf1-ade9-391c364a5a2f admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:01.283202 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-73974f3f-c34f-4bf1-ade9-391c364a5a2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:01.360961 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73974f3f-c34f-4bf1-ade9-391c364a5a2f admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:01.367243 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7660536-2709-4be7-b587-4402840022da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:01.367843 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7660536-2709-4be7-b587-4402840022da service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:01.368123 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7660536-2709-4be7-b587-4402840022da service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:01.368396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7660536-2709-4be7-b587-4402840022da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:01.376025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7660536-2709-4be7-b587-4402840022da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:01.377628 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7660536-2709-4be7-b587-4402840022da service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:01.386870 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1425/2836] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:35:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:35:09.689517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e4067-1101-45f2-b74d-064ce0bd557d None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:09.751527 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e4067-1101-45f2-b74d-064ce0bd557d None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:09.755987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df6c68-818a-488e-9133-aa41975a2c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:09.756606 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df6c68-818a-488e-9133-aa41975a2c5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:09.756914 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df6c68-818a-488e-9133-aa41975a2c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:09.757182 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df6c68-818a-488e-9133-aa41975a2c5c admin admin] PATH_INFO: `/v3/users/464229f012424659878cf01ee92145e0` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:09.758337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df6c68-818a-488e-9133-aa41975a2c5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=464229f012424659878cf01ee92145e0)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:09.760234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df6c68-818a-488e-9133-aa41975a2c5c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:09.771973 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:09.809064 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-28df6c68-818a-488e-9133-aa41975a2c5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '464229f012424659878cf01ee92145e0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:09.815999 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-28df6c68-818a-488e-9133-aa41975a2c5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '464229f012424659878cf01ee92145e0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:09.827631 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-28df6c68-818a-488e-9133-aa41975a2c5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '464229f012424659878cf01ee92145e0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:09.828296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-28df6c68-818a-488e-9133-aa41975a2c5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '464229f012424659878cf01ee92145e0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:09.828911 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-28df6c68-818a-488e-9133-aa41975a2c5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '464229f012424659878cf01ee92145e0'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:09.842242 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1412/2837] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:35:09 2022] DELETE /identity/v3/users/464229f012424659878cf01ee92145e0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:09.846861 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f7c2b-dfc0-4a11-bc36-388fa7adbac9 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:09.847340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f7c2b-dfc0-4a11-bc36-388fa7adbac9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:09.847486 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7f7c2b-dfc0-4a11-bc36-388fa7adbac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:09.866871 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-6b7f7c2b-dfc0-4a11-bc36-388fa7adbac9 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:09.868066 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6b7f7c2b-dfc0-4a11-bc36-388fa7adbac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:09.924387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1426/2838] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:09.934780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df6c68-818a-488e-9133-aa41975a2c5c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:09.936181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-28df6c68-818a-488e-9133-aa41975a2c5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:10.003850 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df6c68-818a-488e-9133-aa41975a2c5c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:10.007424 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e532e6-4a0f-456a-ad80-bae8e0d83cb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:10.008008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e532e6-4a0f-456a-ad80-bae8e0d83cb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:10.008280 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e532e6-4a0f-456a-ad80-bae8e0d83cb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:10.008541 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e532e6-4a0f-456a-ad80-bae8e0d83cb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:10.017174 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e532e6-4a0f-456a-ad80-bae8e0d83cb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:10.018593 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e532e6-4a0f-456a-ad80-bae8e0d83cb0 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:10.028162 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1413/2839] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:35:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:35:10.301324 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f7c2b-dfc0-4a11-bc36-388fa7adbac9 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:10.302783 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-6b7f7c2b-dfc0-4a11-bc36-388fa7adbac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:10.324416 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7f7c2b-dfc0-4a11-bc36-388fa7adbac9 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:10.329311 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9ebaf1-af64-4ce7-9450-171cd58fb9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:10.329958 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ebaf1-af64-4ce7-9450-171cd58fb9fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:10.330253 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ebaf1-af64-4ce7-9450-171cd58fb9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:10.330561 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9ebaf1-af64-4ce7-9450-171cd58fb9fd admin admin] PATH_INFO: `/v3/projects/0a7314d447be4a3cb0bbcb6b82be76a3` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:10.331543 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ebaf1-af64-4ce7-9450-171cd58fb9fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a7314d447be4a3cb0bbcb6b82be76a3)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:10.333479 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9ebaf1-af64-4ce7-9450-171cd58fb9fd admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:10.382734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3d9ebaf1-af64-4ce7-9450-171cd58fb9fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a7314d447be4a3cb0bbcb6b82be76a3', 'request_id': 'req-3d9ebaf1-af64-4ce7-9450-171cd58fb9fd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:10.388732 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3d9ebaf1-af64-4ce7-9450-171cd58fb9fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a7314d447be4a3cb0bbcb6b82be76a3', 'request_id': 'req-3d9ebaf1-af64-4ce7-9450-171cd58fb9fd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:10.394248 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3d9ebaf1-af64-4ce7-9450-171cd58fb9fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a7314d447be4a3cb0bbcb6b82be76a3', 'request_id': 'req-3d9ebaf1-af64-4ce7-9450-171cd58fb9fd'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:10.406524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3d9ebaf1-af64-4ce7-9450-171cd58fb9fd admin admin] The token cache is being invalidate because project 0a7314d447be4a3cb0bbcb6b82be76a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:35:10.406802 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3d9ebaf1-af64-4ce7-9450-171cd58fb9fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:10.407403 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-3d9ebaf1-af64-4ce7-9450-171cd58fb9fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:10.408852 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1427/2840] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:35:10 2022] DELETE /identity/v3/projects/0a7314d447be4a3cb0bbcb6b82be76a3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:10.413763 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf325e-81c2-4165-83c5-5ae8099151ff None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:10.414231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf325e-81c2-4165-83c5-5ae8099151ff None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:10.414574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf325e-81c2-4165-83c5-5ae8099151ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:10.433773 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-edaf325e-81c2-4165-83c5-5ae8099151ff None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:10.434991 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-edaf325e-81c2-4165-83c5-5ae8099151ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:10.492077 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1414/2841] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:10.499913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904a9f4-6b12-450a-918e-00ade4ce16d7 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:10.500194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904a9f4-6b12-450a-918e-00ade4ce16d7 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:10.500462 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0904a9f4-6b12-450a-918e-00ade4ce16d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:10.508581 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:10.523617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0904a9f4-6b12-450a-918e-00ade4ce16d7 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:10.524827 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0904a9f4-6b12-450a-918e-00ade4ce16d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:10.533074 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1428/2842] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:10.537297 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf325e-81c2-4165-83c5-5ae8099151ff None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:10.547145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf325e-81c2-4165-83c5-5ae8099151ff None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:10.551690 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61bb30-349c-48f1-bfbb-ccacc04879e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:10.552280 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61bb30-349c-48f1-bfbb-ccacc04879e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:10.552554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61bb30-349c-48f1-bfbb-ccacc04879e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:10.552822 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61bb30-349c-48f1-bfbb-ccacc04879e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:10.553509 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61bb30-349c-48f1-bfbb-ccacc04879e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:35:10.553808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61bb30-349c-48f1-bfbb-ccacc04879e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:10.555446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61bb30-349c-48f1-bfbb-ccacc04879e8 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:10.566718 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1415/2843] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:35:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:35:10.571427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904a9f4-6b12-450a-918e-00ade4ce16d7 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:10.580930 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0904a9f4-6b12-450a-918e-00ade4ce16d7 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:10.586267 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96299e6b-a7e5-444d-ad77-b6c3e9bff73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:10.586849 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96299e6b-a7e5-444d-ad77-b6c3e9bff73a admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:10.587128 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96299e6b-a7e5-444d-ad77-b6c3e9bff73a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:10.587398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96299e6b-a7e5-444d-ad77-b6c3e9bff73a admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:10.588293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96299e6b-a7e5-444d-ad77-b6c3e9bff73a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:10.590232 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96299e6b-a7e5-444d-ad77-b6c3e9bff73a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:10.616389 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1429/2844] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:35:10 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:35:10.620504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61bb30-349c-48f1-bfbb-ccacc04879e8 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:10.630524 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61bb30-349c-48f1-bfbb-ccacc04879e8 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:10.634747 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:10.689958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd57fb6-3b77-45f5-b289-beb3aa92dc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:10.690594 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd57fb6-3b77-45f5-b289-beb3aa92dc47 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:10.690820 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd57fb6-3b77-45f5-b289-beb3aa92dc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:10.691087 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd57fb6-3b77-45f5-b289-beb3aa92dc47 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:10.691714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd57fb6-3b77-45f5-b289-beb3aa92dc47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:10.693670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd57fb6-3b77-45f5-b289-beb3aa92dc47 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:10.715784 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1416/2845] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:35:10 2022] POST /identity/v3/users => generated 286 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:35:10.719916 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96299e6b-a7e5-444d-ad77-b6c3e9bff73a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:10.730363 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96299e6b-a7e5-444d-ad77-b6c3e9bff73a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:10.734960 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fa1ca-b43d-4f08-ac6a-5e26babc15e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:10.735552 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fa1ca-b43d-4f08-ac6a-5e26babc15e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:10.735831 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fa1ca-b43d-4f08-ac6a-5e26babc15e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:10.736107 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143fa1ca-b43d-4f08-ac6a-5e26babc15e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:10.736733 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fa1ca-b43d-4f08-ac6a-5e26babc15e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:10.738343 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143fa1ca-b43d-4f08-ac6a-5e26babc15e1 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:10.748743 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1430/2846] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:35:10.752551 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd57fb6-3b77-45f5-b289-beb3aa92dc47 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:10.766619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd57fb6-3b77-45f5-b289-beb3aa92dc47 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:10.771472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05d6933-7fe2-4868-bb30-188483f00b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:10.772063 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d6933-7fe2-4868-bb30-188483f00b68 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:10.772332 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d6933-7fe2-4868-bb30-188483f00b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:10.772623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05d6933-7fe2-4868-bb30-188483f00b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:10.773279 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d6933-7fe2-4868-bb30-188483f00b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:10.774843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05d6933-7fe2-4868-bb30-188483f00b68 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:10.792423 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1417/2847] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:35:10 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:35:10.796670 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fa1ca-b43d-4f08-ac6a-5e26babc15e1 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:10.807298 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143fa1ca-b43d-4f08-ac6a-5e26babc15e1 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:10.812119 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc9cc1-aab2-4d5d-8097-7b74c1fcf878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:10.812757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc9cc1-aab2-4d5d-8097-7b74c1fcf878 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:10.813034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc9cc1-aab2-4d5d-8097-7b74c1fcf878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:10.813346 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc9cc1-aab2-4d5d-8097-7b74c1fcf878 admin admin] PATH_INFO: `/v3/projects/5791f243239040e3b8d9600ee4271b62/users/1f90baccf57b42a6b795ce14930e7e86/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:10.828433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc9cc1-aab2-4d5d-8097-7b74c1fcf878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5791f243239040e3b8d9600ee4271b62, user_id=1f90baccf57b42a6b795ce14930e7e86, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:10.831085 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc9cc1-aab2-4d5d-8097-7b74c1fcf878 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:10.844265 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1431/2848] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:35:10 2022] PUT /identity/v3/projects/5791f243239040e3b8d9600ee4271b62/users/1f90baccf57b42a6b795ce14930e7e86/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:10.855952 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bd512-3ba1-4033-958f-3801f4950e84 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:10.856303 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bd512-3ba1-4033-958f-3801f4950e84 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:10.856644 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415bd512-3ba1-4033-958f-3801f4950e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:10.885068 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-415bd512-3ba1-4033-958f-3801f4950e84 None None] MFA Rules not processed for user `1f90baccf57b42a6b795ce14930e7e86`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:10.886234 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-415bd512-3ba1-4033-958f-3801f4950e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:10.948191 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1418/2849] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:10.952669 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc9cc1-aab2-4d5d-8097-7b74c1fcf878 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:11.012536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc9cc1-aab2-4d5d-8097-7b74c1fcf878 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:11.017118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262336a-19fa-4207-9db4-143b7f96d427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:11.017734 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262336a-19fa-4207-9db4-143b7f96d427 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:11.018012 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262336a-19fa-4207-9db4-143b7f96d427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:11.018284 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e262336a-19fa-4207-9db4-143b7f96d427 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:11.018889 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262336a-19fa-4207-9db4-143b7f96d427 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:11.020756 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e262336a-19fa-4207-9db4-143b7f96d427 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:11.046136 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1432/2850] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:35:10 2022] POST /identity/v3/projects => generated 373 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:35:11.050283 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bd512-3ba1-4033-958f-3801f4950e84 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:11.061939 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415bd512-3ba1-4033-958f-3801f4950e84 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:11.119441 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba288016-b38f-4e2c-ab99-44ccdd5f5a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:11.120054 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba288016-b38f-4e2c-ab99-44ccdd5f5a7a admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:11.120339 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba288016-b38f-4e2c-ab99-44ccdd5f5a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:11.120607 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba288016-b38f-4e2c-ab99-44ccdd5f5a7a admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:11.121254 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba288016-b38f-4e2c-ab99-44ccdd5f5a7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:11.123229 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba288016-b38f-4e2c-ab99-44ccdd5f5a7a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:11.144963 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1419/2851] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:35:11 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:35:11.149127 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262336a-19fa-4207-9db4-143b7f96d427 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:11.159642 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e262336a-19fa-4207-9db4-143b7f96d427 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:11.164995 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21687c9f-49ba-4cc8-8302-31340d12fcf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:11.165618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21687c9f-49ba-4cc8-8302-31340d12fcf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:11.165924 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21687c9f-49ba-4cc8-8302-31340d12fcf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:11.166194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21687c9f-49ba-4cc8-8302-31340d12fcf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:11.166825 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21687c9f-49ba-4cc8-8302-31340d12fcf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:11.168434 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21687c9f-49ba-4cc8-8302-31340d12fcf6 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:11.179887 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1433/2852] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:35:11.184351 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba288016-b38f-4e2c-ab99-44ccdd5f5a7a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:11.190957 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:11.195584 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba288016-b38f-4e2c-ab99-44ccdd5f5a7a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:11.202021 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3594d00-cc68-466d-a39a-f6bc77509560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:11.202769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3594d00-cc68-466d-a39a-f6bc77509560 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:11.202948 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3594d00-cc68-466d-a39a-f6bc77509560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:11.203317 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3594d00-cc68-466d-a39a-f6bc77509560 admin admin] PATH_INFO: `/v3/projects/6ee615c894ca4eb0a1917708a617ffae/users/2b7fff1949d8469d88e593d11424fad6/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:11.221570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3594d00-cc68-466d-a39a-f6bc77509560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee615c894ca4eb0a1917708a617ffae, user_id=2b7fff1949d8469d88e593d11424fad6, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:11.224220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3594d00-cc68-466d-a39a-f6bc77509560 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:11.238894 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1420/2853] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:35:11 2022] PUT /identity/v3/projects/6ee615c894ca4eb0a1917708a617ffae/users/2b7fff1949d8469d88e593d11424fad6/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:11.244332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21687c9f-49ba-4cc8-8302-31340d12fcf6 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:11.254240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21687c9f-49ba-4cc8-8302-31340d12fcf6 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:11.311839 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f29bfcf-1873-4f0b-9a28-238bcb558d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:11.313156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29bfcf-1873-4f0b-9a28-238bcb558d16 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:11.313156 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29bfcf-1873-4f0b-9a28-238bcb558d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:11.313388 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f29bfcf-1873-4f0b-9a28-238bcb558d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:11.314094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f29bfcf-1873-4f0b-9a28-238bcb558d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:11.315989 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f29bfcf-1873-4f0b-9a28-238bcb558d16 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:11.327870 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1434/2854] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:35:11 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:35:11.332092 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3594d00-cc68-466d-a39a-f6bc77509560 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:11.344701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3594d00-cc68-466d-a39a-f6bc77509560 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:11.349392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf72e3-de9d-42a2-85c0-a064109bae34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:11.350038 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf72e3-de9d-42a2-85c0-a064109bae34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:11.350312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf72e3-de9d-42a2-85c0-a064109bae34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:11.350585 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccf72e3-de9d-42a2-85c0-a064109bae34 admin admin] PATH_INFO: `/v3/projects/6ee615c894ca4eb0a1917708a617ffae/users/2b7fff1949d8469d88e593d11424fad6/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:11.352484 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf72e3-de9d-42a2-85c0-a064109bae34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ee615c894ca4eb0a1917708a617ffae, user_id=2b7fff1949d8469d88e593d11424fad6, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:11.355142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccf72e3-de9d-42a2-85c0-a064109bae34 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:11.365080 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1421/2855] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:35:11 2022] PUT /identity/v3/projects/6ee615c894ca4eb0a1917708a617ffae/users/2b7fff1949d8469d88e593d11424fad6/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:11.375375 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8edd26-55d6-4933-8c20-9c756cb91a67 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:11.375667 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8edd26-55d6-4933-8c20-9c756cb91a67 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:11.375941 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8edd26-55d6-4933-8c20-9c756cb91a67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:11.395941 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:11.406480 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-9d8edd26-55d6-4933-8c20-9c756cb91a67 None None] MFA Rules not processed for user `2b7fff1949d8469d88e593d11424fad6`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:11.407720 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-9d8edd26-55d6-4933-8c20-9c756cb91a67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:11.474485 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1435/2856] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:11.485663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf72e3-de9d-42a2-85c0-a064109bae34 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:11.487380 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-0ccf72e3-de9d-42a2-85c0-a064109bae34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:11.544589 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccf72e3-de9d-42a2-85c0-a064109bae34 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:11.548217 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3e606b-48a8-441e-980b-37c1e2ffbfa9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:11.548808 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e606b-48a8-441e-980b-37c1e2ffbfa9 service cinder] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:11.549086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e606b-48a8-441e-980b-37c1e2ffbfa9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:11.549391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3e606b-48a8-441e-980b-37c1e2ffbfa9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:11.558739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e606b-48a8-441e-980b-37c1e2ffbfa9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:11.560142 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3e606b-48a8-441e-980b-37c1e2ffbfa9 service cinder] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:11.570139 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1422/2857] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:35:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:35:11.951223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8edd26-55d6-4933-8c20-9c756cb91a67 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:12.016401 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8edd26-55d6-4933-8c20-9c756cb91a67 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:12.021339 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463bf10-5a38-4c54-83c8-72f35be414c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:12.022044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463bf10-5a38-4c54-83c8-72f35be414c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:12.022392 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463bf10-5a38-4c54-83c8-72f35be414c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:12.022742 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2463bf10-5a38-4c54-83c8-72f35be414c9 admin admin] PATH_INFO: `/v3/users/1f90baccf57b42a6b795ce14930e7e86` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:12.023922 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463bf10-5a38-4c54-83c8-72f35be414c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f90baccf57b42a6b795ce14930e7e86)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:12.025935 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2463bf10-5a38-4c54-83c8-72f35be414c9 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:12.073466 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2463bf10-5a38-4c54-83c8-72f35be414c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f90baccf57b42a6b795ce14930e7e86'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:12.080996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2463bf10-5a38-4c54-83c8-72f35be414c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f90baccf57b42a6b795ce14930e7e86'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:12.088234 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2463bf10-5a38-4c54-83c8-72f35be414c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f90baccf57b42a6b795ce14930e7e86'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:12.088887 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2463bf10-5a38-4c54-83c8-72f35be414c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f90baccf57b42a6b795ce14930e7e86'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:12.101568 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-2463bf10-5a38-4c54-83c8-72f35be414c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f90baccf57b42a6b795ce14930e7e86'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:12.107448 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1436/2858] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:35:11 2022] DELETE /identity/v3/users/1f90baccf57b42a6b795ce14930e7e86 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:12.112119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691eb973-f3cc-497a-a446-90ae5583817e None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:12.112598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691eb973-f3cc-497a-a446-90ae5583817e None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:12.112712 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691eb973-f3cc-497a-a446-90ae5583817e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:12.132719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-691eb973-f3cc-497a-a446-90ae5583817e None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:12.133956 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-691eb973-f3cc-497a-a446-90ae5583817e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:12.194139 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1423/2859] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:12.203771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463bf10-5a38-4c54-83c8-72f35be414c9 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:12.205161 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2463bf10-5a38-4c54-83c8-72f35be414c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:12.219867 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:12.266106 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2463bf10-5a38-4c54-83c8-72f35be414c9 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:12.269736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8e3bd-fbc5-4e06-97c9-7818e66db24b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:12.270339 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8e3bd-fbc5-4e06-97c9-7818e66db24b service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:12.270615 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8e3bd-fbc5-4e06-97c9-7818e66db24b service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:12.270885 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a8e3bd-fbc5-4e06-97c9-7818e66db24b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:12.280211 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8e3bd-fbc5-4e06-97c9-7818e66db24b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:12.281673 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a8e3bd-fbc5-4e06-97c9-7818e66db24b service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:12.291487 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1437/2860] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:35:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:35:12.550678 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691eb973-f3cc-497a-a446-90ae5583817e None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:12.552294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-691eb973-f3cc-497a-a446-90ae5583817e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:12.565557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691eb973-f3cc-497a-a446-90ae5583817e None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:12.570589 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55357198-a424-4edb-8374-069cd143947a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:12.571222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55357198-a424-4edb-8374-069cd143947a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:12.571496 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55357198-a424-4edb-8374-069cd143947a admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:12.571764 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55357198-a424-4edb-8374-069cd143947a admin admin] PATH_INFO: `/v3/projects/5791f243239040e3b8d9600ee4271b62` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:12.572903 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55357198-a424-4edb-8374-069cd143947a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5791f243239040e3b8d9600ee4271b62)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:12.575086 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55357198-a424-4edb-8374-069cd143947a admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:12.627391 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-55357198-a424-4edb-8374-069cd143947a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5791f243239040e3b8d9600ee4271b62', 'request_id': 'req-55357198-a424-4edb-8374-069cd143947a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:12.634786 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-55357198-a424-4edb-8374-069cd143947a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5791f243239040e3b8d9600ee4271b62', 'request_id': 'req-55357198-a424-4edb-8374-069cd143947a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:12.640198 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-55357198-a424-4edb-8374-069cd143947a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5791f243239040e3b8d9600ee4271b62', 'request_id': 'req-55357198-a424-4edb-8374-069cd143947a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:12.641334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:12.654023 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-55357198-a424-4edb-8374-069cd143947a admin admin] The token cache is being invalidate because project 5791f243239040e3b8d9600ee4271b62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:35:12.654307 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-55357198-a424-4edb-8374-069cd143947a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:12.654936 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-55357198-a424-4edb-8374-069cd143947a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:12.656405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1424/2861] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:35:12 2022] DELETE /identity/v3/projects/5791f243239040e3b8d9600ee4271b62 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:12.660497 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8e3bd-fbc5-4e06-97c9-7818e66db24b service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:12.670374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a8e3bd-fbc5-4e06-97c9-7818e66db24b service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:12.728332 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d795fc-6b79-4672-9151-f1f84032d36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:12.728882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d795fc-6b79-4672-9151-f1f84032d36e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:12.729160 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d795fc-6b79-4672-9151-f1f84032d36e admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:12.729464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d795fc-6b79-4672-9151-f1f84032d36e admin admin] PATH_INFO: `/v3/users/2b7fff1949d8469d88e593d11424fad6` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:12.730567 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d795fc-6b79-4672-9151-f1f84032d36e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b7fff1949d8469d88e593d11424fad6)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:12.732500 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d795fc-6b79-4672-9151-f1f84032d36e admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:12.778632 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-13d795fc-6b79-4672-9151-f1f84032d36e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b7fff1949d8469d88e593d11424fad6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:12.785374 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-13d795fc-6b79-4672-9151-f1f84032d36e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b7fff1949d8469d88e593d11424fad6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:12.792690 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-13d795fc-6b79-4672-9151-f1f84032d36e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b7fff1949d8469d88e593d11424fad6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:12.793339 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-13d795fc-6b79-4672-9151-f1f84032d36e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b7fff1949d8469d88e593d11424fad6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:12.804983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-13d795fc-6b79-4672-9151-f1f84032d36e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b7fff1949d8469d88e593d11424fad6'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:12.810787 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1438/2862] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:35:12 2022] DELETE /identity/v3/users/2b7fff1949d8469d88e593d11424fad6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:13.073153 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55357198-a424-4edb-8374-069cd143947a admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:13.074763 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-55357198-a424-4edb-8374-069cd143947a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:13.088321 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55357198-a424-4edb-8374-069cd143947a admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:13.143440 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5509f9b2-c7c3-4a35-a395-50adf68dbad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:13.144135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509f9b2-c7c3-4a35-a395-50adf68dbad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:13.144313 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509f9b2-c7c3-4a35-a395-50adf68dbad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:13.144579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5509f9b2-c7c3-4a35-a395-50adf68dbad9 admin admin] PATH_INFO: `/v3/projects/6ee615c894ca4eb0a1917708a617ffae` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:13.145678 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509f9b2-c7c3-4a35-a395-50adf68dbad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ee615c894ca4eb0a1917708a617ffae)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:13.147588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5509f9b2-c7c3-4a35-a395-50adf68dbad9 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:13.192683 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5509f9b2-c7c3-4a35-a395-50adf68dbad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ee615c894ca4eb0a1917708a617ffae', 'request_id': 'req-5509f9b2-c7c3-4a35-a395-50adf68dbad9'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:13.201405 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5509f9b2-c7c3-4a35-a395-50adf68dbad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ee615c894ca4eb0a1917708a617ffae', 'request_id': 'req-5509f9b2-c7c3-4a35-a395-50adf68dbad9'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:13.207073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5509f9b2-c7c3-4a35-a395-50adf68dbad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ee615c894ca4eb0a1917708a617ffae', 'request_id': 'req-5509f9b2-c7c3-4a35-a395-50adf68dbad9'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:13.218410 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5509f9b2-c7c3-4a35-a395-50adf68dbad9 admin admin] The token cache is being invalidate because project 6ee615c894ca4eb0a1917708a617ffae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81627) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:35:13.218696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5509f9b2-c7c3-4a35-a395-50adf68dbad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:13.219337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-5509f9b2-c7c3-4a35-a395-50adf68dbad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:13.220816 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1425/2863] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:35:13 2022] DELETE /identity/v3/projects/6ee615c894ca4eb0a1917708a617ffae => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:13.225849 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddc302-2dd2-4f3a-bd74-eb4026f104f9 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:13.226357 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddc302-2dd2-4f3a-bd74-eb4026f104f9 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:13.226554 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fddc302-2dd2-4f3a-bd74-eb4026f104f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:13.246429 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-0fddc302-2dd2-4f3a-bd74-eb4026f104f9 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:13.247618 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0fddc302-2dd2-4f3a-bd74-eb4026f104f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:13.277860 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:13.308983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1439/2864] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:13.316843 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256643c3-8cd0-41a6-b213-a99045b83891 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:13.317146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256643c3-8cd0-41a6-b213-a99045b83891 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:13.317436 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256643c3-8cd0-41a6-b213-a99045b83891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:13.337114 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-256643c3-8cd0-41a6-b213-a99045b83891 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:13.338581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-256643c3-8cd0-41a6-b213-a99045b83891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:13.346247 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1426/2865] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:13.350293 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddc302-2dd2-4f3a-bd74-eb4026f104f9 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:13.360912 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fddc302-2dd2-4f3a-bd74-eb4026f104f9 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:13.365511 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ff989-e8d7-4e11-bdb0-0dd5fdc2e84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:13.366093 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ff989-e8d7-4e11-bdb0-0dd5fdc2e84a admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:13.367001 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ff989-e8d7-4e11-bdb0-0dd5fdc2e84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:13.367268 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ff989-e8d7-4e11-bdb0-0dd5fdc2e84a admin admin] PATH_INFO: `/v3/domains` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:13.367899 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ff989-e8d7-4e11-bdb0-0dd5fdc2e84a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81626) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:35:13.368194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ff989-e8d7-4e11-bdb0-0dd5fdc2e84a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:13.369822 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ff989-e8d7-4e11-bdb0-0dd5fdc2e84a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:13.381268 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1440/2866] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:35:13.385612 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256643c3-8cd0-41a6-b213-a99045b83891 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:13.394780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256643c3-8cd0-41a6-b213-a99045b83891 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:13.399368 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636ff31-6342-4544-95d7-30621c1d67a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:13.399949 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636ff31-6342-4544-95d7-30621c1d67a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:13.400221 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636ff31-6342-4544-95d7-30621c1d67a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:13.400486 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7636ff31-6342-4544-95d7-30621c1d67a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:13.401094 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636ff31-6342-4544-95d7-30621c1d67a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:13.402989 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7636ff31-6342-4544-95d7-30621c1d67a0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:13.430446 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1427/2867] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:35:13 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:35:13.434512 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ff989-e8d7-4e11-bdb0-0dd5fdc2e84a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:13.444258 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ff989-e8d7-4e11-bdb0-0dd5fdc2e84a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:13.499070 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636a420-67b5-4108-9a64-68dd2e35a6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:13.499656 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636a420-67b5-4108-9a64-68dd2e35a6eb admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:13.499930 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636a420-67b5-4108-9a64-68dd2e35a6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:13.500196 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636a420-67b5-4108-9a64-68dd2e35a6eb admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:13.500800 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636a420-67b5-4108-9a64-68dd2e35a6eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:13.502710 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c636a420-67b5-4108-9a64-68dd2e35a6eb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:13.523494 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1441/2868] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:35:13 2022] POST /identity/v3/users => generated 294 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:35:13.527564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636ff31-6342-4544-95d7-30621c1d67a0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:13.537580 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7636ff31-6342-4544-95d7-30621c1d67a0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:13.542564 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba3754-963d-412c-b0c2-5b21d8984b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:13.543145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba3754-963d-412c-b0c2-5b21d8984b16 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:13.543413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba3754-963d-412c-b0c2-5b21d8984b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:13.543678 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba3754-963d-412c-b0c2-5b21d8984b16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:13.544301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba3754-963d-412c-b0c2-5b21d8984b16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:13.545906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba3754-963d-412c-b0c2-5b21d8984b16 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:13.556101 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1428/2869] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:35:13.559854 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636a420-67b5-4108-9a64-68dd2e35a6eb admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:13.569705 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c636a420-67b5-4108-9a64-68dd2e35a6eb admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:13.574471 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ab79de-0f6a-4f89-a51d-f4a1b67411ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:13.575221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab79de-0f6a-4f89-a51d-f4a1b67411ff admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:13.575502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab79de-0f6a-4f89-a51d-f4a1b67411ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:13.575773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ab79de-0f6a-4f89-a51d-f4a1b67411ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:13.576407 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab79de-0f6a-4f89-a51d-f4a1b67411ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:13.578103 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ab79de-0f6a-4f89-a51d-f4a1b67411ff admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:13.589485 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1442/2870] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:35:13.593719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba3754-963d-412c-b0c2-5b21d8984b16 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:13.603946 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba3754-963d-412c-b0c2-5b21d8984b16 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:13.609015 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d58cd2-865d-4cb8-81c4-3d95c28808f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:13.609670 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d58cd2-865d-4cb8-81c4-3d95c28808f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:13.609941 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d58cd2-865d-4cb8-81c4-3d95c28808f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:13.610206 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d58cd2-865d-4cb8-81c4-3d95c28808f5 admin admin] PATH_INFO: `/v3/projects/d74616ca2a8744298db453e77f9a76f1/users/9992ffec516a484a83fa58847bf0c5d5/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:13.624638 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d58cd2-865d-4cb8-81c4-3d95c28808f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d74616ca2a8744298db453e77f9a76f1, user_id=9992ffec516a484a83fa58847bf0c5d5, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:13.627257 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d58cd2-865d-4cb8-81c4-3d95c28808f5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:13.640598 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1429/2871] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:35:13 2022] PUT /identity/v3/projects/d74616ca2a8744298db453e77f9a76f1/users/9992ffec516a484a83fa58847bf0c5d5/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:35:13.651918 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef7a4d-7db9-4eca-bd70-1a68663241b4 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:13.652196 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef7a4d-7db9-4eca-bd70-1a68663241b4 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:13.652464 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feef7a4d-7db9-4eca-bd70-1a68663241b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:13.681022 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-feef7a4d-7db9-4eca-bd70-1a68663241b4 None None] MFA Rules not processed for user `9992ffec516a484a83fa58847bf0c5d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:13.682319 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-feef7a4d-7db9-4eca-bd70-1a68663241b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:13.745726 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1443/2872] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:13 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:13.750199 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d58cd2-865d-4cb8-81c4-3d95c28808f5 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:13.770729 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:13.815460 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d58cd2-865d-4cb8-81c4-3d95c28808f5 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:13.820117 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f8890-aa8c-4d7f-a0ba-d178b15b806d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:13.820720 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f8890-aa8c-4d7f-a0ba-d178b15b806d admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:13.820984 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f8890-aa8c-4d7f-a0ba-d178b15b806d admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:13.821282 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8f8890-aa8c-4d7f-a0ba-d178b15b806d admin admin] PATH_INFO: `/v3/projects` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:13.821910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f8890-aa8c-4d7f-a0ba-d178b15b806d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:13.823785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8f8890-aa8c-4d7f-a0ba-d178b15b806d admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:13.849682 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1430/2873] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:35:13 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 31 19:35:13.854383 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef7a4d-7db9-4eca-bd70-1a68663241b4 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:13.864996 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feef7a4d-7db9-4eca-bd70-1a68663241b4 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:13.892992 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:13.922393 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b5bc9-095b-447d-bda5-67d76c6dc77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:13.922988 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b5bc9-095b-447d-bda5-67d76c6dc77a admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:13.923260 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b5bc9-095b-447d-bda5-67d76c6dc77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:13.923524 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859b5bc9-095b-447d-bda5-67d76c6dc77a admin admin] PATH_INFO: `/v3/users` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:13.924137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b5bc9-095b-447d-bda5-67d76c6dc77a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:13.926048 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859b5bc9-095b-447d-bda5-67d76c6dc77a admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:13.946411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1444/2874] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:35:13 2022] POST /identity/v3/users => generated 300 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:35:13.950434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f8890-aa8c-4d7f-a0ba-d178b15b806d admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:13.961329 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8f8890-aa8c-4d7f-a0ba-d178b15b806d admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:13.966289 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56465d60-767b-430f-a323-4bf0f50004c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:13.966873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56465d60-767b-430f-a323-4bf0f50004c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:13.967148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56465d60-767b-430f-a323-4bf0f50004c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:13.967417 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56465d60-767b-430f-a323-4bf0f50004c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:13.968047 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56465d60-767b-430f-a323-4bf0f50004c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:13.969666 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56465d60-767b-430f-a323-4bf0f50004c0 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:13.981059 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1431/2875] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:35:13.985179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b5bc9-095b-447d-bda5-67d76c6dc77a admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:13.995128 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859b5bc9-095b-447d-bda5-67d76c6dc77a admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:13.999728 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11312bdd-c795-43a4-ba73-67de0745e2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:14.000369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11312bdd-c795-43a4-ba73-67de0745e2eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:14.000649 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11312bdd-c795-43a4-ba73-67de0745e2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:14.000937 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11312bdd-c795-43a4-ba73-67de0745e2eb admin admin] PATH_INFO: `/v3/projects/0800fa26a0044f4297929a0f023adc07/users/be0ed8fc27a643ba8865b34232576b5e/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:14.015194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11312bdd-c795-43a4-ba73-67de0745e2eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=0800fa26a0044f4297929a0f023adc07, user_id=be0ed8fc27a643ba8865b34232576b5e, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:14.018004 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11312bdd-c795-43a4-ba73-67de0745e2eb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:14.031358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1445/2876] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:35:13 2022] PUT /identity/v3/projects/0800fa26a0044f4297929a0f023adc07/users/be0ed8fc27a643ba8865b34232576b5e/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:14.035379 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56465d60-767b-430f-a323-4bf0f50004c0 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:14.045838 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56465d60-767b-430f-a323-4bf0f50004c0 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:14.103231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee494f3-9b1d-4641-a0ac-384026493f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:14.103757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee494f3-9b1d-4641-a0ac-384026493f79 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:14.104037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee494f3-9b1d-4641-a0ac-384026493f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:14.104309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee494f3-9b1d-4641-a0ac-384026493f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:14.104943 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee494f3-9b1d-4641-a0ac-384026493f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:14.106570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee494f3-9b1d-4641-a0ac-384026493f79 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:14.116222 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1432/2877] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:35:14.120587 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11312bdd-c795-43a4-ba73-67de0745e2eb admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:14.130735 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11312bdd-c795-43a4-ba73-67de0745e2eb admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:14.136030 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dc443-3c0a-4988-b6e2-c5b7d5f57222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:14.136655 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dc443-3c0a-4988-b6e2-c5b7d5f57222 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:14.136926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dc443-3c0a-4988-b6e2-c5b7d5f57222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:14.137209 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7dc443-3c0a-4988-b6e2-c5b7d5f57222 admin admin] PATH_INFO: `/v3/projects/0800fa26a0044f4297929a0f023adc07/users/be0ed8fc27a643ba8865b34232576b5e/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:14.139135 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7dc443-3c0a-4988-b6e2-c5b7d5f57222 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0800fa26a0044f4297929a0f023adc07, user_id=be0ed8fc27a643ba8865b34232576b5e, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:14.141912 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7dc443-3c0a-4988-b6e2-c5b7d5f57222 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:14.152781 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1446/2878] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:35:14 2022] PUT /identity/v3/projects/0800fa26a0044f4297929a0f023adc07/users/be0ed8fc27a643ba8865b34232576b5e/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:14.162660 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e9865-6ee0-4caa-bf19-bcb73fbc0107 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:14.163143 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e9865-6ee0-4caa-bf19-bcb73fbc0107 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:14.163244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1e9865-6ee0-4caa-bf19-bcb73fbc0107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:14.190386 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-ec1e9865-6ee0-4caa-bf19-bcb73fbc0107 None None] MFA Rules not processed for user `be0ed8fc27a643ba8865b34232576b5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:14.191589 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-ec1e9865-6ee0-4caa-bf19-bcb73fbc0107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:14.260255 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1433/2879] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:14 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:14.271016 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dc443-3c0a-4988-b6e2-c5b7d5f57222 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:14.273474 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ea7dc443-3c0a-4988-b6e2-c5b7d5f57222 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:14.328757 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7dc443-3c0a-4988-b6e2-c5b7d5f57222 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:14.332320 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18158a3a-d0b3-42ba-818a-831dceeaeff7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:14.332907 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18158a3a-d0b3-42ba-818a-831dceeaeff7 service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:14.333179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18158a3a-d0b3-42ba-818a-831dceeaeff7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:14.333478 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18158a3a-d0b3-42ba-818a-831dceeaeff7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:14.343046 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18158a3a-d0b3-42ba-818a-831dceeaeff7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:14.344450 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18158a3a-d0b3-42ba-818a-831dceeaeff7 service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:14.353668 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1447/2880] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:35:14 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:35:14.889049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e9865-6ee0-4caa-bf19-bcb73fbc0107 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:14.905882 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:14.952135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1e9865-6ee0-4caa-bf19-bcb73fbc0107 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:14.956576 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c431e-5d51-42fc-9d85-a0856130a8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:14.957204 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c431e-5d51-42fc-9d85-a0856130a8fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:14.957494 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c431e-5d51-42fc-9d85-a0856130a8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:14.957759 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c431e-5d51-42fc-9d85-a0856130a8fc admin admin] PATH_INFO: `/v3/users/9992ffec516a484a83fa58847bf0c5d5` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:14.958853 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c431e-5d51-42fc-9d85-a0856130a8fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9992ffec516a484a83fa58847bf0c5d5)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:14.960739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c431e-5d51-42fc-9d85-a0856130a8fc admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:15.006326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7a9c431e-5d51-42fc-9d85-a0856130a8fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9992ffec516a484a83fa58847bf0c5d5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:15.013135 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7a9c431e-5d51-42fc-9d85-a0856130a8fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9992ffec516a484a83fa58847bf0c5d5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:15.024752 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7a9c431e-5d51-42fc-9d85-a0856130a8fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9992ffec516a484a83fa58847bf0c5d5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:15.025434 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7a9c431e-5d51-42fc-9d85-a0856130a8fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9992ffec516a484a83fa58847bf0c5d5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:15.026049 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-7a9c431e-5d51-42fc-9d85-a0856130a8fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9992ffec516a484a83fa58847bf0c5d5'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:15.038744 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1434/2881] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:35:14 2022] DELETE /identity/v3/users/9992ffec516a484a83fa58847bf0c5d5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:15.043625 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adf4c8-3a2b-4a3c-8d08-dfb3701c5987 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:15.044000 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adf4c8-3a2b-4a3c-8d08-dfb3701c5987 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:15.044358 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45adf4c8-3a2b-4a3c-8d08-dfb3701c5987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:15.064411 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-45adf4c8-3a2b-4a3c-8d08-dfb3701c5987 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:15.065630 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-45adf4c8-3a2b-4a3c-8d08-dfb3701c5987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:15.124098 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1448/2882] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:15.134122 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c431e-5d51-42fc-9d85-a0856130a8fc admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:15.135518 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-7a9c431e-5d51-42fc-9d85-a0856130a8fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:15.191230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c431e-5d51-42fc-9d85-a0856130a8fc admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:15.195520 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0c937-7b84-4b11-8cf4-7d5208668666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:15.196225 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0c937-7b84-4b11-8cf4-7d5208668666 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:15.196378 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0c937-7b84-4b11-8cf4-7d5208668666 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:15.196643 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af0c937-7b84-4b11-8cf4-7d5208668666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:15.207630 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0c937-7b84-4b11-8cf4-7d5208668666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:15.209288 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af0c937-7b84-4b11-8cf4-7d5208668666 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:15.223490 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1435/2883] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:35:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:35:15.496577 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adf4c8-3a2b-4a3c-8d08-dfb3701c5987 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:15.498077 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-45adf4c8-3a2b-4a3c-8d08-dfb3701c5987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:15.510799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45adf4c8-3a2b-4a3c-8d08-dfb3701c5987 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:15.515339 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96f709-7386-453a-9a70-5def3c9e476b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:15.515959 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96f709-7386-453a-9a70-5def3c9e476b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:15.516234 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96f709-7386-453a-9a70-5def3c9e476b admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:15.516501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e96f709-7386-453a-9a70-5def3c9e476b admin admin] PATH_INFO: `/v3/projects/d74616ca2a8744298db453e77f9a76f1` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:15.517640 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96f709-7386-453a-9a70-5def3c9e476b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d74616ca2a8744298db453e77f9a76f1)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:15.519553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e96f709-7386-453a-9a70-5def3c9e476b admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:15.559280 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:15.583441 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7e96f709-7386-453a-9a70-5def3c9e476b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd74616ca2a8744298db453e77f9a76f1', 'request_id': 'req-7e96f709-7386-453a-9a70-5def3c9e476b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:15.590060 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7e96f709-7386-453a-9a70-5def3c9e476b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd74616ca2a8744298db453e77f9a76f1', 'request_id': 'req-7e96f709-7386-453a-9a70-5def3c9e476b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:15.595789 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7e96f709-7386-453a-9a70-5def3c9e476b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd74616ca2a8744298db453e77f9a76f1', 'request_id': 'req-7e96f709-7386-453a-9a70-5def3c9e476b'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:15.609476 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7e96f709-7386-453a-9a70-5def3c9e476b admin admin] The token cache is being invalidate because project d74616ca2a8744298db453e77f9a76f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:35:15.609764 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7e96f709-7386-453a-9a70-5def3c9e476b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:15.610427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-7e96f709-7386-453a-9a70-5def3c9e476b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:15.611910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1449/2884] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:35:15 2022] DELETE /identity/v3/projects/d74616ca2a8744298db453e77f9a76f1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:15.615902 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0c937-7b84-4b11-8cf4-7d5208668666 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:15.626119 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af0c937-7b84-4b11-8cf4-7d5208668666 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:15.657244 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:15.682806 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c442ad28-7e4a-4b98-bb77-d9b9f4f87ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:15.683429 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442ad28-7e4a-4b98-bb77-d9b9f4f87ab6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:15.683709 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442ad28-7e4a-4b98-bb77-d9b9f4f87ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:15.683982 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c442ad28-7e4a-4b98-bb77-d9b9f4f87ab6 admin admin] PATH_INFO: `/v3/users/be0ed8fc27a643ba8865b34232576b5e` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:15.685103 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442ad28-7e4a-4b98-bb77-d9b9f4f87ab6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be0ed8fc27a643ba8865b34232576b5e)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:15.687036 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c442ad28-7e4a-4b98-bb77-d9b9f4f87ab6 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:15.732454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c442ad28-7e4a-4b98-bb77-d9b9f4f87ab6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be0ed8fc27a643ba8865b34232576b5e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:15.739207 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c442ad28-7e4a-4b98-bb77-d9b9f4f87ab6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be0ed8fc27a643ba8865b34232576b5e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:15.751302 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c442ad28-7e4a-4b98-bb77-d9b9f4f87ab6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be0ed8fc27a643ba8865b34232576b5e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:15.751958 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c442ad28-7e4a-4b98-bb77-d9b9f4f87ab6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be0ed8fc27a643ba8865b34232576b5e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:15.752561 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-c442ad28-7e4a-4b98-bb77-d9b9f4f87ab6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be0ed8fc27a643ba8865b34232576b5e'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:15.764760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1436/2885] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:35:15 2022] DELETE /identity/v3/users/be0ed8fc27a643ba8865b34232576b5e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:16.032346 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96f709-7386-453a-9a70-5def3c9e476b admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:16.033879 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-7e96f709-7386-453a-9a70-5def3c9e476b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:16.047276 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e96f709-7386-453a-9a70-5def3c9e476b admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:16.104783 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d8236-ceb6-4c95-8114-feefd6c33d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:16.105438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d8236-ceb6-4c95-8114-feefd6c33d47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:16.105716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d8236-ceb6-4c95-8114-feefd6c33d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:16.105983 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d8236-ceb6-4c95-8114-feefd6c33d47 admin admin] PATH_INFO: `/v3/projects/0800fa26a0044f4297929a0f023adc07` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:16.107099 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d8236-ceb6-4c95-8114-feefd6c33d47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0800fa26a0044f4297929a0f023adc07)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:16.110373 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d8236-ceb6-4c95-8114-feefd6c33d47 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:16.160536 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0c3d8236-ceb6-4c95-8114-feefd6c33d47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0800fa26a0044f4297929a0f023adc07', 'request_id': 'req-0c3d8236-ceb6-4c95-8114-feefd6c33d47'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:16.167047 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0c3d8236-ceb6-4c95-8114-feefd6c33d47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0800fa26a0044f4297929a0f023adc07', 'request_id': 'req-0c3d8236-ceb6-4c95-8114-feefd6c33d47'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:16.172459 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0c3d8236-ceb6-4c95-8114-feefd6c33d47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0800fa26a0044f4297929a0f023adc07', 'request_id': 'req-0c3d8236-ceb6-4c95-8114-feefd6c33d47'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:16.184327 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0c3d8236-ceb6-4c95-8114-feefd6c33d47 admin admin] The token cache is being invalidate because project 0800fa26a0044f4297929a0f023adc07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:35:16.184605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0c3d8236-ceb6-4c95-8114-feefd6c33d47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:16.185228 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-0c3d8236-ceb6-4c95-8114-feefd6c33d47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:35:16.186678 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1450/2886] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:35:16 2022] DELETE /identity/v3/projects/0800fa26a0044f4297929a0f023adc07 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:16.191649 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b9c3a-e50a-46f7-bd88-21d81ccaff43 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:16.192146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b9c3a-e50a-46f7-bd88-21d81ccaff43 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:16.192517 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b9c3a-e50a-46f7-bd88-21d81ccaff43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:16.213037 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-a94b9c3a-e50a-46f7-bd88-21d81ccaff43 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:16.214429 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-a94b9c3a-e50a-46f7-bd88-21d81ccaff43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:16.273241 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1437/2887] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:16.281213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90ef4b-457a-46cf-a663-89c4f77fe521 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:16.281502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90ef4b-457a-46cf-a663-89c4f77fe521 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:16.281773 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef90ef4b-457a-46cf-a663-89c4f77fe521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:16.301432 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-ef90ef4b-457a-46cf-a663-89c4f77fe521 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:16.302636 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-ef90ef4b-457a-46cf-a663-89c4f77fe521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:16.310720 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1451/2888] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:16.314930 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b9c3a-e50a-46f7-bd88-21d81ccaff43 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:16.324579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b9c3a-e50a-46f7-bd88-21d81ccaff43 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:16.329059 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990eaeb-c141-4f05-81cb-09450e3cb513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:16.329691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990eaeb-c141-4f05-81cb-09450e3cb513 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:16.329966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990eaeb-c141-4f05-81cb-09450e3cb513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:16.330232 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6990eaeb-c141-4f05-81cb-09450e3cb513 admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:16.330861 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990eaeb-c141-4f05-81cb-09450e3cb513 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:35:16.331156 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990eaeb-c141-4f05-81cb-09450e3cb513 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:16.332727 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6990eaeb-c141-4f05-81cb-09450e3cb513 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:16.344757 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1438/2889] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:35:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:35:16.349055 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90ef4b-457a-46cf-a663-89c4f77fe521 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:16.358767 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef90ef4b-457a-46cf-a663-89c4f77fe521 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:16.363329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389d28d-5695-435b-80c4-244ab5db6adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:16.363908 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389d28d-5695-435b-80c4-244ab5db6adb admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:16.364194 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389d28d-5695-435b-80c4-244ab5db6adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:16.364447 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f389d28d-5695-435b-80c4-244ab5db6adb admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:16.365062 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389d28d-5695-435b-80c4-244ab5db6adb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:16.366978 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f389d28d-5695-435b-80c4-244ab5db6adb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:16.394640 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1452/2890] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:35:16 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:35:16.399428 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990eaeb-c141-4f05-81cb-09450e3cb513 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:16.409239 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6990eaeb-c141-4f05-81cb-09450e3cb513 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:16.449443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:16.466864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30d447-9ec3-41a7-a55d-831b290dfa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:16.467484 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30d447-9ec3-41a7-a55d-831b290dfa98 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:16.467760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30d447-9ec3-41a7-a55d-831b290dfa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:16.468031 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30d447-9ec3-41a7-a55d-831b290dfa98 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:16.468656 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30d447-9ec3-41a7-a55d-831b290dfa98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:16.470581 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de30d447-9ec3-41a7-a55d-831b290dfa98 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:16.490286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1439/2891] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:35:16 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:35:16.494414 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389d28d-5695-435b-80c4-244ab5db6adb admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:16.504418 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f389d28d-5695-435b-80c4-244ab5db6adb admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:16.509484 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a44bee-c898-481d-b972-644c4dfe4964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:16.510068 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a44bee-c898-481d-b972-644c4dfe4964 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:16.510342 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a44bee-c898-481d-b972-644c4dfe4964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:16.510609 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a44bee-c898-481d-b972-644c4dfe4964 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:16.511240 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a44bee-c898-481d-b972-644c4dfe4964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:16.512799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a44bee-c898-481d-b972-644c4dfe4964 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:16.517999 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:16.524421 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1453/2892] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:35:16.528325 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30d447-9ec3-41a7-a55d-831b290dfa98 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:16.538173 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30d447-9ec3-41a7-a55d-831b290dfa98 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:16.542789 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decc6645-f529-4717-875c-e706f70b0755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:16.543370 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc6645-f529-4717-875c-e706f70b0755 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:16.543645 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc6645-f529-4717-875c-e706f70b0755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:16.543912 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decc6645-f529-4717-875c-e706f70b0755 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:16.544537 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc6645-f529-4717-875c-e706f70b0755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:16.546126 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decc6645-f529-4717-875c-e706f70b0755 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:16.556378 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1440/2893] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:35:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:35:16.560353 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a44bee-c898-481d-b972-644c4dfe4964 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:16.570574 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a44bee-c898-481d-b972-644c4dfe4964 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:16.575187 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae6733-94dc-4be2-868a-51b25f19c058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:16.575824 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae6733-94dc-4be2-868a-51b25f19c058 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:16.576102 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae6733-94dc-4be2-868a-51b25f19c058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:16.576376 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae6733-94dc-4be2-868a-51b25f19c058 admin admin] PATH_INFO: `/v3/projects/3253764fbd7541d2bdbaeb70a6715c2f/users/79fa83baed7847a29086b7f0da96828a/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:16.591592 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae6733-94dc-4be2-868a-51b25f19c058 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3253764fbd7541d2bdbaeb70a6715c2f, user_id=79fa83baed7847a29086b7f0da96828a, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:16.594303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae6733-94dc-4be2-868a-51b25f19c058 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:16.606391 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1454/2894] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:35:16 2022] PUT /identity/v3/projects/3253764fbd7541d2bdbaeb70a6715c2f/users/79fa83baed7847a29086b7f0da96828a/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:35:16.617591 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ca249-e671-423c-b514-1f45e02547e8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:16.617873 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ca249-e671-423c-b514-1f45e02547e8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:16.618148 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8ca249-e671-423c-b514-1f45e02547e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:16.645415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-1e8ca249-e671-423c-b514-1f45e02547e8 None None] MFA Rules not processed for user `79fa83baed7847a29086b7f0da96828a`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:35:16.646619 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1e8ca249-e671-423c-b514-1f45e02547e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:16.709080 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1441/2895] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:35:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:35:16.721589 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae6733-94dc-4be2-868a-51b25f19c058 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:35:16.723335 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-48ae6733-94dc-4be2-868a-51b25f19c058 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:35:16.798964 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae6733-94dc-4be2-868a-51b25f19c058 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:35:16.802542 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cb28eb-13e0-4799-9037-38c24106f2a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1863336fecad4987acdfd0b3f7b2f70f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:35:16.803130 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb28eb-13e0-4799-9037-38c24106f2a9 service glance] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:35:16.803403 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb28eb-13e0-4799-9037-38c24106f2a9 service glance] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:35:16.804155 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cb28eb-13e0-4799-9037-38c24106f2a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:35:16.812144 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb28eb-13e0-4799-9037-38c24106f2a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:35:16.813566 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cb28eb-13e0-4799-9037-38c24106f2a9 service glance] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:35:16.822633 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1455/2896] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:35:16 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:35:26.706146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=46,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:35:26.818185 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=45,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:01.864069 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ca249-e671-423c-b514-1f45e02547e8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:01.924826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8ca249-e671-423c-b514-1f45e02547e8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:01.929340 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43feec58-2542-4501-a443-b1f8c1d86c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:01.929954 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43feec58-2542-4501-a443-b1f8c1d86c38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:01.930235 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43feec58-2542-4501-a443-b1f8c1d86c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:01.930504 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43feec58-2542-4501-a443-b1f8c1d86c38 admin admin] PATH_INFO: `/v3/users/79fa83baed7847a29086b7f0da96828a` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:01.931604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43feec58-2542-4501-a443-b1f8c1d86c38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79fa83baed7847a29086b7f0da96828a)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:01.934071 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43feec58-2542-4501-a443-b1f8c1d86c38 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:01.979853 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-43feec58-2542-4501-a443-b1f8c1d86c38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79fa83baed7847a29086b7f0da96828a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:01.987197 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-43feec58-2542-4501-a443-b1f8c1d86c38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79fa83baed7847a29086b7f0da96828a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:01.999418 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-43feec58-2542-4501-a443-b1f8c1d86c38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79fa83baed7847a29086b7f0da96828a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:02.000084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-43feec58-2542-4501-a443-b1f8c1d86c38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79fa83baed7847a29086b7f0da96828a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:02.000688 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-43feec58-2542-4501-a443-b1f8c1d86c38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79fa83baed7847a29086b7f0da96828a'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:02.012901 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1442/2897] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:36:01 2022] DELETE /identity/v3/users/79fa83baed7847a29086b7f0da96828a => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:02.017395 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdc61b-f12c-4ed7-83a7-a9c0638c8143 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:02.017926 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdc61b-f12c-4ed7-83a7-a9c0638c8143 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:02.018044 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdc61b-f12c-4ed7-83a7-a9c0638c8143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:02.037014 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-46fdc61b-f12c-4ed7-83a7-a9c0638c8143 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:02.038230 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-46fdc61b-f12c-4ed7-83a7-a9c0638c8143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:02.096615 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1456/2898] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:02.106401 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43feec58-2542-4501-a443-b1f8c1d86c38 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:02.107789 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-43feec58-2542-4501-a443-b1f8c1d86c38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:02.163788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43feec58-2542-4501-a443-b1f8c1d86c38 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:02.167472 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58baf9e-d370-4d55-a3fc-55397d786346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:02.168120 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58baf9e-d370-4d55-a3fc-55397d786346 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:02.168393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58baf9e-d370-4d55-a3fc-55397d786346 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:02.168658 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58baf9e-d370-4d55-a3fc-55397d786346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:02.177468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58baf9e-d370-4d55-a3fc-55397d786346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:02.178867 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58baf9e-d370-4d55-a3fc-55397d786346 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:02.188154 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1443/2899] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:02.454065 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdc61b-f12c-4ed7-83a7-a9c0638c8143 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:02.455496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-46fdc61b-f12c-4ed7-83a7-a9c0638c8143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:02.467659 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdc61b-f12c-4ed7-83a7-a9c0638c8143 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:02.472676 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3c3933-e455-44d3-a133-7ef2067e16e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:02.473330 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c3933-e455-44d3-a133-7ef2067e16e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:02.473624 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c3933-e455-44d3-a133-7ef2067e16e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:02.473898 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3c3933-e455-44d3-a133-7ef2067e16e9 admin admin] PATH_INFO: `/v3/projects/3253764fbd7541d2bdbaeb70a6715c2f` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:02.475034 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c3933-e455-44d3-a133-7ef2067e16e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3253764fbd7541d2bdbaeb70a6715c2f)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:02.476868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3c3933-e455-44d3-a133-7ef2067e16e9 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:02.525449 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-fe3c3933-e455-44d3-a133-7ef2067e16e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3253764fbd7541d2bdbaeb70a6715c2f', 'request_id': 'req-fe3c3933-e455-44d3-a133-7ef2067e16e9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:02.531573 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-fe3c3933-e455-44d3-a133-7ef2067e16e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3253764fbd7541d2bdbaeb70a6715c2f', 'request_id': 'req-fe3c3933-e455-44d3-a133-7ef2067e16e9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:02.536938 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-fe3c3933-e455-44d3-a133-7ef2067e16e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3253764fbd7541d2bdbaeb70a6715c2f', 'request_id': 'req-fe3c3933-e455-44d3-a133-7ef2067e16e9'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:02.549327 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-fe3c3933-e455-44d3-a133-7ef2067e16e9 admin admin] The token cache is being invalidate because project 3253764fbd7541d2bdbaeb70a6715c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:36:02.549611 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-fe3c3933-e455-44d3-a133-7ef2067e16e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:02.550221 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-fe3c3933-e455-44d3-a133-7ef2067e16e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:02.551707 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1457/2900] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:36:02 2022] DELETE /identity/v3/projects/3253764fbd7541d2bdbaeb70a6715c2f => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:02.556536 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbc3bf-46fd-45c8-96d1-807ec39b968b None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:02.556811 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbc3bf-46fd-45c8-96d1-807ec39b968b None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:02.557080 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbc3bf-46fd-45c8-96d1-807ec39b968b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:02.577995 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-36cbc3bf-46fd-45c8-96d1-807ec39b968b None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:02.579196 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-36cbc3bf-46fd-45c8-96d1-807ec39b968b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:02.595910 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:02.639260 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1444/2901] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:02.647203 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316aee8-cf86-42ac-b12c-6d7c4994dd10 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:02.647513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316aee8-cf86-42ac-b12c-6d7c4994dd10 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:02.647786 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8316aee8-cf86-42ac-b12c-6d7c4994dd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:02.668019 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-8316aee8-cf86-42ac-b12c-6d7c4994dd10 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:02.669241 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-8316aee8-cf86-42ac-b12c-6d7c4994dd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:02.677238 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1458/2902] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:36:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:02.681450 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbc3bf-46fd-45c8-96d1-807ec39b968b None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:02.691833 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbc3bf-46fd-45c8-96d1-807ec39b968b None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:02.696337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7771e3-a0d0-44f9-a238-57f41c21aceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:02.696927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7771e3-a0d0-44f9-a238-57f41c21aceb admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:02.697231 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7771e3-a0d0-44f9-a238-57f41c21aceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:02.697508 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7771e3-a0d0-44f9-a238-57f41c21aceb admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:02.698145 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7771e3-a0d0-44f9-a238-57f41c21aceb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:36:02.698447 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7771e3-a0d0-44f9-a238-57f41c21aceb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:02.700589 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7771e3-a0d0-44f9-a238-57f41c21aceb admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:02.712230 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1445/2903] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:36:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 31 19:36:02.716944 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316aee8-cf86-42ac-b12c-6d7c4994dd10 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:02.726329 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8316aee8-cf86-42ac-b12c-6d7c4994dd10 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:02.732227 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61593043-0a3a-4999-9c48-ce8f6c5fe6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:02.732814 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61593043-0a3a-4999-9c48-ce8f6c5fe6eb admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:02.733109 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61593043-0a3a-4999-9c48-ce8f6c5fe6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:02.733401 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61593043-0a3a-4999-9c48-ce8f6c5fe6eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:02.734055 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61593043-0a3a-4999-9c48-ce8f6c5fe6eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:02.735946 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61593043-0a3a-4999-9c48-ce8f6c5fe6eb admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:02.762639 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1459/2904] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:36:02 2022] POST /identity/v3/projects => generated 397 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:02.766797 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7771e3-a0d0-44f9-a238-57f41c21aceb admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:02.777657 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7771e3-a0d0-44f9-a238-57f41c21aceb admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:02.832335 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761efec-874b-4c06-b2b5-77dfc578ee27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:02.832957 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761efec-874b-4c06-b2b5-77dfc578ee27 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:02.833260 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761efec-874b-4c06-b2b5-77dfc578ee27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:02.833534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761efec-874b-4c06-b2b5-77dfc578ee27 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:02.834149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761efec-874b-4c06-b2b5-77dfc578ee27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:02.836084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761efec-874b-4c06-b2b5-77dfc578ee27 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:02.857208 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1446/2905] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:36:02 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:02.861365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61593043-0a3a-4999-9c48-ce8f6c5fe6eb admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:02.871678 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61593043-0a3a-4999-9c48-ce8f6c5fe6eb admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:02.876239 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e71fb-7219-4bff-8158-687ef8450aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:02.876827 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e71fb-7219-4bff-8158-687ef8450aab admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:02.877102 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e71fb-7219-4bff-8158-687ef8450aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:02.877396 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8e71fb-7219-4bff-8158-687ef8450aab admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:02.878023 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e71fb-7219-4bff-8158-687ef8450aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:02.879597 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8e71fb-7219-4bff-8158-687ef8450aab admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:02.890599 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1460/2906] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:02.894291 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761efec-874b-4c06-b2b5-77dfc578ee27 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:02.904782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761efec-874b-4c06-b2b5-77dfc578ee27 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:02.909420 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea44379-8769-4c52-91ed-1a24f5b96d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:02.910008 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea44379-8769-4c52-91ed-1a24f5b96d7f admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:02.910286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea44379-8769-4c52-91ed-1a24f5b96d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:02.910559 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea44379-8769-4c52-91ed-1a24f5b96d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:02.911184 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea44379-8769-4c52-91ed-1a24f5b96d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:02.912746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea44379-8769-4c52-91ed-1a24f5b96d7f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:02.923007 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1447/2907] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:02.926984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e71fb-7219-4bff-8158-687ef8450aab admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:02.936982 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8e71fb-7219-4bff-8158-687ef8450aab admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:02.941810 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b617076-a358-4809-8420-926f12d58235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:02.942442 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b617076-a358-4809-8420-926f12d58235 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:02.942724 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b617076-a358-4809-8420-926f12d58235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:02.942997 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b617076-a358-4809-8420-926f12d58235 admin admin] PATH_INFO: `/v3/projects/51a550a973b2465fbdf36538385fe0ee/users/9f43ed0998e84d5484bb09155e38f747/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:02.957922 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b617076-a358-4809-8420-926f12d58235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51a550a973b2465fbdf36538385fe0ee, user_id=9f43ed0998e84d5484bb09155e38f747, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:02.960525 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b617076-a358-4809-8420-926f12d58235 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:02.973984 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1461/2908] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:36:02 2022] PUT /identity/v3/projects/51a550a973b2465fbdf36538385fe0ee/users/9f43ed0998e84d5484bb09155e38f747/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:02.985210 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5eb17-0dfa-4a66-ad7a-3d8d4fe3c1f8 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:02.985496 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5eb17-0dfa-4a66-ad7a-3d8d4fe3c1f8 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:02.985769 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5eb17-0dfa-4a66-ad7a-3d8d4fe3c1f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:03.012974 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9fb5eb17-0dfa-4a66-ad7a-3d8d4fe3c1f8 None None] MFA Rules not processed for user `9f43ed0998e84d5484bb09155e38f747`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:03.014209 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9fb5eb17-0dfa-4a66-ad7a-3d8d4fe3c1f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:03.078270 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1448/2909] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:36:02 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:03.090375 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b617076-a358-4809-8420-926f12d58235 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:03.092043 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5b617076-a358-4809-8420-926f12d58235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:03.113868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:03.149241 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b617076-a358-4809-8420-926f12d58235 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:03.153269 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073a5646-f0c2-46fd-a6df-f17ff2f46bbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:03.153868 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a5646-f0c2-46fd-a6df-f17ff2f46bbe service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:03.154143 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a5646-f0c2-46fd-a6df-f17ff2f46bbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:03.154413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073a5646-f0c2-46fd-a6df-f17ff2f46bbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:03.162533 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a5646-f0c2-46fd-a6df-f17ff2f46bbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:03.163940 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073a5646-f0c2-46fd-a6df-f17ff2f46bbe service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:03.173249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1462/2910] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:03 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:11.224767 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5eb17-0dfa-4a66-ad7a-3d8d4fe3c1f8 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:11.243970 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:11.287951 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5eb17-0dfa-4a66-ad7a-3d8d4fe3c1f8 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:11.292502 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262e709-7e5f-409c-ad01-0abdc0de3c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:11.293118 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262e709-7e5f-409c-ad01-0abdc0de3c78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:11.293422 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262e709-7e5f-409c-ad01-0abdc0de3c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:11.293691 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262e709-7e5f-409c-ad01-0abdc0de3c78 admin admin] PATH_INFO: `/v3/users/9f43ed0998e84d5484bb09155e38f747` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:11.294780 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262e709-7e5f-409c-ad01-0abdc0de3c78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f43ed0998e84d5484bb09155e38f747)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:11.296777 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262e709-7e5f-409c-ad01-0abdc0de3c78 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:11.345782 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f262e709-7e5f-409c-ad01-0abdc0de3c78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f43ed0998e84d5484bb09155e38f747'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:11.352647 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f262e709-7e5f-409c-ad01-0abdc0de3c78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f43ed0998e84d5484bb09155e38f747'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:11.364393 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f262e709-7e5f-409c-ad01-0abdc0de3c78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f43ed0998e84d5484bb09155e38f747'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:11.365055 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f262e709-7e5f-409c-ad01-0abdc0de3c78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f43ed0998e84d5484bb09155e38f747'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:11.365687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-f262e709-7e5f-409c-ad01-0abdc0de3c78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f43ed0998e84d5484bb09155e38f747'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:11.378695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1449/2911] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:36:11 2022] DELETE /identity/v3/users/9f43ed0998e84d5484bb09155e38f747 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:11.383274 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4a6ed-aa71-41ab-b62c-f8dd94132760 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:11.383788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4a6ed-aa71-41ab-b62c-f8dd94132760 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:11.383884 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c4a6ed-aa71-41ab-b62c-f8dd94132760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:11.403115 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-e8c4a6ed-aa71-41ab-b62c-f8dd94132760 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:11.404398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e8c4a6ed-aa71-41ab-b62c-f8dd94132760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:11.463059 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1463/2912] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:36:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:11.472704 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262e709-7e5f-409c-ad01-0abdc0de3c78 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:11.474218 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-f262e709-7e5f-409c-ad01-0abdc0de3c78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:11.530088 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262e709-7e5f-409c-ad01-0abdc0de3c78 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:11.533752 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7bd7fc-a6d8-4590-babf-450d1f444e0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:11.534339 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bd7fc-a6d8-4590-babf-450d1f444e0b service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:11.534610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bd7fc-a6d8-4590-babf-450d1f444e0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:11.534876 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7bd7fc-a6d8-4590-babf-450d1f444e0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:11.543574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7bd7fc-a6d8-4590-babf-450d1f444e0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:11.545018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7bd7fc-a6d8-4590-babf-450d1f444e0b service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:11.554286 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1450/2913] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:11.817889 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4a6ed-aa71-41ab-b62c-f8dd94132760 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:11.819338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e8c4a6ed-aa71-41ab-b62c-f8dd94132760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:11.831948 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c4a6ed-aa71-41ab-b62c-f8dd94132760 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:11.837525 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa5d4c-cb65-4ba8-8d15-bd475f76d667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:11.838162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa5d4c-cb65-4ba8-8d15-bd475f76d667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:11.838443 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa5d4c-cb65-4ba8-8d15-bd475f76d667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:11.838716 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa5d4c-cb65-4ba8-8d15-bd475f76d667 admin admin] PATH_INFO: `/v3/projects/51a550a973b2465fbdf36538385fe0ee` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:11.840051 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa5d4c-cb65-4ba8-8d15-bd475f76d667 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51a550a973b2465fbdf36538385fe0ee)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:11.842086 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebaa5d4c-cb65-4ba8-8d15-bd475f76d667 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:11.890852 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebaa5d4c-cb65-4ba8-8d15-bd475f76d667 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51a550a973b2465fbdf36538385fe0ee', 'request_id': 'req-ebaa5d4c-cb65-4ba8-8d15-bd475f76d667'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:11.896808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebaa5d4c-cb65-4ba8-8d15-bd475f76d667 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51a550a973b2465fbdf36538385fe0ee', 'request_id': 'req-ebaa5d4c-cb65-4ba8-8d15-bd475f76d667'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:11.902340 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebaa5d4c-cb65-4ba8-8d15-bd475f76d667 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51a550a973b2465fbdf36538385fe0ee', 'request_id': 'req-ebaa5d4c-cb65-4ba8-8d15-bd475f76d667'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:11.914116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebaa5d4c-cb65-4ba8-8d15-bd475f76d667 admin admin] The token cache is being invalidate because project 51a550a973b2465fbdf36538385fe0ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:36:11.914398 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebaa5d4c-cb65-4ba8-8d15-bd475f76d667 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:11.915022 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-ebaa5d4c-cb65-4ba8-8d15-bd475f76d667 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:36:11.916496 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1464/2914] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:36:11 2022] DELETE /identity/v3/projects/51a550a973b2465fbdf36538385fe0ee => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:11.922073 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baed09-facf-411b-9859-09331a035134 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:11.922371 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baed09-facf-411b-9859-09331a035134 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:11.922641 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35baed09-facf-411b-9859-09331a035134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:11.942412 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-35baed09-facf-411b-9859-09331a035134 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:11.943588 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-35baed09-facf-411b-9859-09331a035134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:11.978294 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:12.005468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1451/2915] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:36:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:12.013502 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0b1a9-fa2c-4c2a-a876-46bfb410d767 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:12.013792 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0b1a9-fa2c-4c2a-a876-46bfb410d767 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:12.014071 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0b1a9-fa2c-4c2a-a876-46bfb410d767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:12.034060 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-70b0b1a9-fa2c-4c2a-a876-46bfb410d767 None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:12.035309 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-70b0b1a9-fa2c-4c2a-a876-46bfb410d767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:12.043534 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1465/2916] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:12.048212 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35baed09-facf-411b-9859-09331a035134 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:12.058018 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35baed09-facf-411b-9859-09331a035134 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:12.062586 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8d363-c363-45b5-a932-05e2d25e3a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:12.063301 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8d363-c363-45b5-a932-05e2d25e3a4b admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:12.063574 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8d363-c363-45b5-a932-05e2d25e3a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:12.063840 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e8d363-c363-45b5-a932-05e2d25e3a4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:12.064468 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8d363-c363-45b5-a932-05e2d25e3a4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81627) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 31 19:36:12.064760 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8d363-c363-45b5-a932-05e2d25e3a4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:12.066361 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e8d363-c363-45b5-a932-05e2d25e3a4b admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:12.080867 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1452/2917] 10.222.0.65 () {58 vars in 1153 bytes} [Wed Aug 31 19:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 31 19:36:12.085366 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0b1a9-fa2c-4c2a-a876-46bfb410d767 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:12.095570 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0b1a9-fa2c-4c2a-a876-46bfb410d767 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:12.100267 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed73000-7014-451c-a49a-0629c960d743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:12.100855 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed73000-7014-451c-a49a-0629c960d743 admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:12.101134 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed73000-7014-451c-a49a-0629c960d743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:12.101438 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed73000-7014-451c-a49a-0629c960d743 admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:12.102069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed73000-7014-451c-a49a-0629c960d743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:12.103966 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed73000-7014-451c-a49a-0629c960d743 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:12.130925 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1466/2918] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:36:12 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:12.135181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8d363-c363-45b5-a932-05e2d25e3a4b admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:12.144990 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e8d363-c363-45b5-a932-05e2d25e3a4b admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:12.219813 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29835ab-e352-4416-b512-24442e1c5b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:12.220413 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29835ab-e352-4416-b512-24442e1c5b4c admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:12.220696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29835ab-e352-4416-b512-24442e1c5b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:12.220953 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29835ab-e352-4416-b512-24442e1c5b4c admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:12.221617 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29835ab-e352-4416-b512-24442e1c5b4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:12.223557 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29835ab-e352-4416-b512-24442e1c5b4c admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:12.246944 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1453/2919] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:36:12 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:12.250808 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed73000-7014-451c-a49a-0629c960d743 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:12.261420 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed73000-7014-451c-a49a-0629c960d743 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:12.266444 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784333f3-21f8-4e5f-b80f-5a882661d751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:12.267063 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784333f3-21f8-4e5f-b80f-5a882661d751 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:12.267341 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784333f3-21f8-4e5f-b80f-5a882661d751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:12.267641 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784333f3-21f8-4e5f-b80f-5a882661d751 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:12.268296 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784333f3-21f8-4e5f-b80f-5a882661d751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:12.270069 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784333f3-21f8-4e5f-b80f-5a882661d751 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:12.281542 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1467/2920] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:12.285663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29835ab-e352-4416-b512-24442e1c5b4c admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:12.296111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29835ab-e352-4416-b512-24442e1c5b4c admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:12.301334 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3da201-6598-49b9-9afc-9049c6850bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:12.302007 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3da201-6598-49b9-9afc-9049c6850bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:12.302456 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3da201-6598-49b9-9afc-9049c6850bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:12.302795 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3da201-6598-49b9-9afc-9049c6850bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:12.303513 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3da201-6598-49b9-9afc-9049c6850bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:12.305306 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3da201-6598-49b9-9afc-9049c6850bc3 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:12.316646 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1454/2921] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:12.320867 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784333f3-21f8-4e5f-b80f-5a882661d751 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:12.331285 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784333f3-21f8-4e5f-b80f-5a882661d751 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:12.336238 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858e3b2-223a-492e-a4eb-7363b3443718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:12.337151 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858e3b2-223a-492e-a4eb-7363b3443718 admin admin] REQUEST_METHOD: `PUT` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:12.337461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858e3b2-223a-492e-a4eb-7363b3443718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:12.337736 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858e3b2-223a-492e-a4eb-7363b3443718 admin admin] PATH_INFO: `/v3/projects/2f7614eeafc645958e7308586df90b8e/users/8983f4ff039e4f6bbf01159accb1fb7c/roles/b80a0d1a8d164b53bef5726b92e68676` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:12.340932 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:12.357365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858e3b2-223a-492e-a4eb-7363b3443718 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f7614eeafc645958e7308586df90b8e, user_id=8983f4ff039e4f6bbf01159accb1fb7c, role_id=b80a0d1a8d164b53bef5726b92e68676)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:12.361687 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858e3b2-223a-492e-a4eb-7363b3443718 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:12.375656 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1468/2922] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:36:12 2022] PUT /identity/v3/projects/2f7614eeafc645958e7308586df90b8e/users/8983f4ff039e4f6bbf01159accb1fb7c/roles/b80a0d1a8d164b53bef5726b92e68676 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:12.380937 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20055972-3a49-47e1-b1da-a852f3fee4eb None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:12.381266 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20055972-3a49-47e1-b1da-a852f3fee4eb None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:12.381539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20055972-3a49-47e1-b1da-a852f3fee4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:12.401610 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-20055972-3a49-47e1-b1da-a852f3fee4eb None None] MFA Rules not processed for user `2d817dbb81e84447ac0c66c6621a6e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:12.402799 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-20055972-3a49-47e1-b1da-a852f3fee4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:12.476579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1455/2923] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:12.486435 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858e3b2-223a-492e-a4eb-7363b3443718 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:12.488153 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2858e3b2-223a-492e-a4eb-7363b3443718 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:12.543749 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858e3b2-223a-492e-a4eb-7363b3443718 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:12.548562 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e6ea-306d-4ff9-b466-2901411abf86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:12.549176 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e6ea-306d-4ff9-b466-2901411abf86 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:12.549488 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e6ea-306d-4ff9-b466-2901411abf86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:12.549759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6e6ea-306d-4ff9-b466-2901411abf86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:12.559166 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6e6ea-306d-4ff9-b466-2901411abf86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:12.560586 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6e6ea-306d-4ff9-b466-2901411abf86 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:12.572325 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1469/2924] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:16.362456 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca3688-18d7-4dd4-8661-f86fa294a369 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:16.363038 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca3688-18d7-4dd4-8661-f86fa294a369 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:16.363165 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ca3688-18d7-4dd4-8661-f86fa294a369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:16.394356 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-63ca3688-18d7-4dd4-8661-f86fa294a369 None None] MFA Rules not processed for user `8983f4ff039e4f6bbf01159accb1fb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:16.395595 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-63ca3688-18d7-4dd4-8661-f86fa294a369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:16.430926 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:16.464903 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1456/2925] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:36:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:16.469777 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e6ea-306d-4ff9-b466-2901411abf86 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:16.479588 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6e6ea-306d-4ff9-b466-2901411abf86 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:16.484817 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843267a9-4943-4d5c-a9af-cd8efc00266d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:16.485448 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843267a9-4943-4d5c-a9af-cd8efc00266d admin admin] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:16.485730 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843267a9-4943-4d5c-a9af-cd8efc00266d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:16.486004 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843267a9-4943-4d5c-a9af-cd8efc00266d admin admin] PATH_INFO: `/v3/projects` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:16.486631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843267a9-4943-4d5c-a9af-cd8efc00266d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:16.488553 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843267a9-4943-4d5c-a9af-cd8efc00266d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:16.519863 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1470/2926] 10.222.0.65 () {60 vars in 1153 bytes} [Wed Aug 31 19:36:16 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:16.521392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca3688-18d7-4dd4-8661-f86fa294a369 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:16.533044 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ca3688-18d7-4dd4-8661-f86fa294a369 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:16.593539 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08bee7-1774-45c4-b2da-b88facad32b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:16.594149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08bee7-1774-45c4-b2da-b88facad32b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:16.594427 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08bee7-1774-45c4-b2da-b88facad32b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:16.594696 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08bee7-1774-45c4-b2da-b88facad32b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:16.595323 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08bee7-1774-45c4-b2da-b88facad32b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:16.597264 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08bee7-1774-45c4-b2da-b88facad32b5 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:16.620403 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1457/2927] 10.222.0.65 () {60 vars in 1144 bytes} [Wed Aug 31 19:36:16 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 31 19:36:16.624604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843267a9-4943-4d5c-a9af-cd8efc00266d admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:16.635559 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843267a9-4943-4d5c-a9af-cd8efc00266d admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:16.640913 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb1cd0-b494-4269-95d2-f1987df33733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:16.641489 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb1cd0-b494-4269-95d2-f1987df33733 admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:16.641771 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb1cd0-b494-4269-95d2-f1987df33733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:16.642045 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fb1cd0-b494-4269-95d2-f1987df33733 admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:16.642689 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb1cd0-b494-4269-95d2-f1987df33733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:16.644301 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fb1cd0-b494-4269-95d2-f1987df33733 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:16.656191 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1471/2928] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 31 19:36:16.660406 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08bee7-1774-45c4-b2da-b88facad32b5 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:16.672415 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08bee7-1774-45c4-b2da-b88facad32b5 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:16.677045 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ecd42-3c06-4ecd-a753-bb9ca2be671f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:16.677766 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ecd42-3c06-4ecd-a753-bb9ca2be671f admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:16.678038 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ecd42-3c06-4ecd-a753-bb9ca2be671f admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:16.678303 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80ecd42-3c06-4ecd-a753-bb9ca2be671f admin admin] PATH_INFO: `/v3/projects/33383c2ac29b4582a33f096cfb798d57/users/94784a7747294ced93016b7a7595ad86/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:16.694130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ecd42-3c06-4ecd-a753-bb9ca2be671f admin admin] RBAC: Authorizing `identity:create_grant(project_id=33383c2ac29b4582a33f096cfb798d57, user_id=94784a7747294ced93016b7a7595ad86, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:16.696750 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80ecd42-3c06-4ecd-a753-bb9ca2be671f admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:16.712611 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1458/2929] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:36:16 2022] PUT /identity/v3/projects/33383c2ac29b4582a33f096cfb798d57/users/94784a7747294ced93016b7a7595ad86/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:16.716663 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb1cd0-b494-4269-95d2-f1987df33733 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:16.727878 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fb1cd0-b494-4269-95d2-f1987df33733 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:16.785796 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bd279-23cc-4008-bd80-b9e0c8365c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:16.786382 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bd279-23cc-4008-bd80-b9e0c8365c9d admin admin] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:16.786661 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bd279-23cc-4008-bd80-b9e0c8365c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:16.786935 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bd279-23cc-4008-bd80-b9e0c8365c9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:16.787558 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90bd279-23cc-4008-bd80-b9e0c8365c9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:16.789126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90bd279-23cc-4008-bd80-b9e0c8365c9d admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:16.799245 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1472/2930] 10.222.0.65 () {58 vars in 1122 bytes} [Wed Aug 31 19:36:16 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 31 19:36:16.803398 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ecd42-3c06-4ecd-a753-bb9ca2be671f admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:16.813538 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80ecd42-3c06-4ecd-a753-bb9ca2be671f admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:16.818083 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2bb6a0-fc41-4b92-ada1-d2ea6f02bd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:16.818716 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bb6a0-fc41-4b92-ada1-d2ea6f02bd96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:16.818987 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bb6a0-fc41-4b92-ada1-d2ea6f02bd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:16.819251 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2bb6a0-fc41-4b92-ada1-d2ea6f02bd96 admin admin] PATH_INFO: `/v3/projects/33383c2ac29b4582a33f096cfb798d57/users/94784a7747294ced93016b7a7595ad86/roles/3580c894dd974d35bc95848b8ba22598` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:16.821111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2bb6a0-fc41-4b92-ada1-d2ea6f02bd96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33383c2ac29b4582a33f096cfb798d57, user_id=94784a7747294ced93016b7a7595ad86, role_id=3580c894dd974d35bc95848b8ba22598)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:16.823714 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2bb6a0-fc41-4b92-ada1-d2ea6f02bd96 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:16.833804 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1459/2931] 10.222.0.65 () {60 vars in 1483 bytes} [Wed Aug 31 19:36:16 2022] PUT /identity/v3/projects/33383c2ac29b4582a33f096cfb798d57/users/94784a7747294ced93016b7a7595ad86/roles/3580c894dd974d35bc95848b8ba22598 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:36:21.199694 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a574af5-2f16-45ca-bacd-d7184b64d92c None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:21.200223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a574af5-2f16-45ca-bacd-d7184b64d92c None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:21.200223 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a574af5-2f16-45ca-bacd-d7184b64d92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:21.239114 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-5a574af5-2f16-45ca-bacd-d7184b64d92c None None] MFA Rules not processed for user `94784a7747294ced93016b7a7595ad86`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:21.240330 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-5a574af5-2f16-45ca-bacd-d7184b64d92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:21.318998 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:21.372907 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1473/2932] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:36:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:21.395701 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2bb6a0-fc41-4b92-ada1-d2ea6f02bd96 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:21.397679 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-fe2bb6a0-fc41-4b92-ada1-d2ea6f02bd96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:21.486124 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2bb6a0-fc41-4b92-ada1-d2ea6f02bd96 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:21.492270 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b2a1f-cba9-4430-9ab6-dfbbc879fe51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:21.492919 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b2a1f-cba9-4430-9ab6-dfbbc879fe51 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:21.493325 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b2a1f-cba9-4430-9ab6-dfbbc879fe51 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:21.493663 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b2a1f-cba9-4430-9ab6-dfbbc879fe51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:21.521002 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:21.556118 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b2a1f-cba9-4430-9ab6-dfbbc879fe51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:21.557774 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b2a1f-cba9-4430-9ab6-dfbbc879fe51 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:21.571833 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1460/2933] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:31.363788 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=12 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:31.564627 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=16 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:45.230513 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116bb9a-9a38-48fd-991c-2a14b78d8048 None None] REQUEST_METHOD: `POST` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:45.231092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116bb9a-9a38-48fd-991c-2a14b78d8048 None None] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:45.231092 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2116bb9a-9a38-48fd-991c-2a14b78d8048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:45.254799 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.auth.core [None req-2116bb9a-9a38-48fd-991c-2a14b78d8048 None None] MFA Rules not processed for user `8983f4ff039e4f6bbf01159accb1fb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81626) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:36:45.256305 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2116bb9a-9a38-48fd-991c-2a14b78d8048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:45.264202 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1474/2934] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:36:45 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:36:45.846325 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b2a1f-cba9-4430-9ab6-dfbbc879fe51 service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:45.847834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-796b2a1f-cba9-4430-9ab6-dfbbc879fe51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:45.903985 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b2a1f-cba9-4430-9ab6-dfbbc879fe51 service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:45.907794 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732b89d-9127-49b9-849d-9a682ab5d5b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:45.908395 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732b89d-9127-49b9-849d-9a682ab5d5b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:45.908695 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732b89d-9127-49b9-849d-9a682ab5d5b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:45.908966 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5732b89d-9127-49b9-849d-9a682ab5d5b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:45.910713 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5732b89d-9127-49b9-849d-9a682ab5d5b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:45.920206 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732b89d-9127-49b9-849d-9a682ab5d5b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:45.921662 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5732b89d-9127-49b9-849d-9a682ab5d5b4 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:45.929596 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1461/2935] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:46.391509 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116bb9a-9a38-48fd-991c-2a14b78d8048 None None] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:46.393094 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-2116bb9a-9a38-48fd-991c-2a14b78d8048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:46.463506 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2116bb9a-9a38-48fd-991c-2a14b78d8048 None None] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:46.468490 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6dc49-73d3-45f8-bbcc-a31941ee3f7b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:46.469089 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6dc49-73d3-45f8-bbcc-a31941ee3f7b service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:46.469412 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6dc49-73d3-45f8-bbcc-a31941ee3f7b service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:46.469686 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6dc49-73d3-45f8-bbcc-a31941ee3f7b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:46.471428 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-3ca6dc49-73d3-45f8-bbcc-a31941ee3f7b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:46.483365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6dc49-73d3-45f8-bbcc-a31941ee3f7b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:46.484988 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca6dc49-73d3-45f8-bbcc-a31941ee3f7b service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:46.494911 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1475/2936] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:46.972755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732b89d-9127-49b9-849d-9a682ab5d5b4 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:36:46.975337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-5732b89d-9127-49b9-849d-9a682ab5d5b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:46.989690 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5732b89d-9127-49b9-849d-9a682ab5d5b4 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:36:46.994623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7ae40-152e-4c1a-a343-632c02f8009a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:36:46.995212 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7ae40-152e-4c1a-a343-632c02f8009a service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:36:46.995491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7ae40-152e-4c1a-a343-632c02f8009a service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:36:46.995758 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc7ae40-152e-4c1a-a343-632c02f8009a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:36:46.997477 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bdc7ae40-152e-4c1a-a343-632c02f8009a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:36:47.010577 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7ae40-152e-4c1a-a343-632c02f8009a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:36:47.012149 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc7ae40-152e-4c1a-a343-632c02f8009a service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:36:47.022402 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1462/2937] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:36:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:36:56.490696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=35 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:36:57.017890 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=32 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:37:04.229605 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6dc49-73d3-45f8-bbcc-a31941ee3f7b service placement] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:04.239025 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6dc49-73d3-45f8-bbcc-a31941ee3f7b service placement] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:04.242567 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbb4b5-4953-42fb-be7a-fff9a1fd90fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:04.243157 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbb4b5-4953-42fb-be7a-fff9a1fd90fd service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:04.243433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbb4b5-4953-42fb-be7a-fff9a1fd90fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:04.243699 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cbb4b5-4953-42fb-be7a-fff9a1fd90fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:04.252613 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cbb4b5-4953-42fb-be7a-fff9a1fd90fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:04.254102 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cbb4b5-4953-42fb-be7a-fff9a1fd90fd service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:04.263532 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1476/2938] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:37:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:37:06.040183 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7ae40-152e-4c1a-a343-632c02f8009a service placement] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:37:06.041719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-bdc7ae40-152e-4c1a-a343-632c02f8009a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:06.057554 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc7ae40-152e-4c1a-a343-632c02f8009a service placement] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:37:06.063003 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47540f56-46b8-4905-9dba-1ed92a3d1000 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:37:06.063595 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47540f56-46b8-4905-9dba-1ed92a3d1000 service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:37:06.063878 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47540f56-46b8-4905-9dba-1ed92a3d1000 service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:37:06.064147 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47540f56-46b8-4905-9dba-1ed92a3d1000 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:37:06.065869 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-47540f56-46b8-4905-9dba-1ed92a3d1000 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:37:06.078060 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47540f56-46b8-4905-9dba-1ed92a3d1000 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:37:06.079625 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47540f56-46b8-4905-9dba-1ed92a3d1000 service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:37:06.091228 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1463/2939] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:37:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:37:14.259369 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=6 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:37:16.086346 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=6 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:38:41.843797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbb4b5-4953-42fb-be7a-fff9a1fd90fd service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:38:41.853383 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cbb4b5-4953-42fb-be7a-fff9a1fd90fd service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:38:41.856943 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376afe45-4895-4ce7-9ec4-c2bc6ea0c0ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:38:41.857570 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376afe45-4895-4ce7-9ec4-c2bc6ea0c0ee service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:38:41.857978 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376afe45-4895-4ce7-9ec4-c2bc6ea0c0ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:38:41.858247 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376afe45-4895-4ce7-9ec4-c2bc6ea0c0ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:38:41.860229 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-376afe45-4895-4ce7-9ec4-c2bc6ea0c0ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:38:41.870250 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376afe45-4895-4ce7-9ec4-c2bc6ea0c0ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:38:41.871656 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376afe45-4895-4ce7-9ec4-c2bc6ea0c0ee service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:38:41.879832 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1477/2940] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:38:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 31 19:38:51.876303 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=6 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:39:23.110623 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47540f56-46b8-4905-9dba-1ed92a3d1000 service nova] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:23.120104 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47540f56-46b8-4905-9dba-1ed92a3d1000 service nova] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:23.124366 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea1a36-820a-4434-a0b7-8825c887f7e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:23.124978 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea1a36-820a-4434-a0b7-8825c887f7e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:23.125276 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea1a36-820a-4434-a0b7-8825c887f7e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:23.125546 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea1a36-820a-4434-a0b7-8825c887f7e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:23.127337 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-88ea1a36-820a-4434-a0b7-8825c887f7e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:23.140482 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea1a36-820a-4434-a0b7-8825c887f7e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:23.141975 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea1a36-820a-4434-a0b7-8825c887f7e9 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:23.152627 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1464/2941] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:39:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:39:26.108423 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376afe45-4895-4ce7-9ec4-c2bc6ea0c0ee service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:26.123418 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376afe45-4895-4ce7-9ec4-c2bc6ea0c0ee service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:26.129103 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c2a5b-e945-49b7-b0f9-f2acbc11421c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:26.129850 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c2a5b-e945-49b7-b0f9-f2acbc11421c service nova] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:26.130233 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c2a5b-e945-49b7-b0f9-f2acbc11421c service nova] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:26.130617 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c2a5b-e945-49b7-b0f9-f2acbc11421c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:26.132470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-db1c2a5b-e945-49b7-b0f9-f2acbc11421c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:26.147387 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c2a5b-e945-49b7-b0f9-f2acbc11421c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:26.149079 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c2a5b-e945-49b7-b0f9-f2acbc11421c service nova] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:26.161863 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1478/2942] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:39:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:39:27.890076 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211dca9-6c52-42ae-8d79-0f8949739349 None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:27.890570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211dca9-6c52-42ae-8d79-0f8949739349 None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:27.890692 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4211dca9-6c52-42ae-8d79-0f8949739349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:27.908870 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-4211dca9-6c52-42ae-8d79-0f8949739349 None None] MFA Rules not processed for user `8983f4ff039e4f6bbf01159accb1fb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:39:27.910083 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-4211dca9-6c52-42ae-8d79-0f8949739349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:27.916927 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1465/2943] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:39:27 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:39:36.156162 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=6 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:39:37.903571 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=9 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:39:39.080608 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c2a5b-e945-49b7-b0f9-f2acbc11421c service nova] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:39.090932 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c2a5b-e945-49b7-b0f9-f2acbc11421c service nova] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:39.094501 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf46898-afd3-47d6-b618-16685ff02f11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:39.095118 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf46898-afd3-47d6-b618-16685ff02f11 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:39.095401 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf46898-afd3-47d6-b618-16685ff02f11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:39.095677 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf46898-afd3-47d6-b618-16685ff02f11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:39.097427 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0bf46898-afd3-47d6-b618-16685ff02f11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:39.111121 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf46898-afd3-47d6-b618-16685ff02f11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:39.112523 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf46898-afd3-47d6-b618-16685ff02f11 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:39.123461 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1479/2944] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:39:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:39:42.062110 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211dca9-6c52-42ae-8d79-0f8949739349 None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:39:42.076084 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4211dca9-6c52-42ae-8d79-0f8949739349 None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:39:42.081719 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d55f0fb-906c-446b-94d3-88a2ffdc663d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '673a72ac9a9847e3812007b66ac7a5ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'ResellerAdmin', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:39:42.082326 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55f0fb-906c-446b-94d3-88a2ffdc663d service nova] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:39:42.082630 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55f0fb-906c-446b-94d3-88a2ffdc663d service nova] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:39:42.082897 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d55f0fb-906c-446b-94d3-88a2ffdc663d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:39:42.084879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2d55f0fb-906c-446b-94d3-88a2ffdc663d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:39:42.098004 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55f0fb-906c-446b-94d3-88a2ffdc663d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:39:42.099560 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d55f0fb-906c-446b-94d3-88a2ffdc663d service nova] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:39:42.110856 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1466/2945] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:39:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:39:49.118537 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=6 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:39:52.106300 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=6 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:41:35.561706 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf46898-afd3-47d6-b618-16685ff02f11 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:41:35.563147 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-0bf46898-afd3-47d6-b618-16685ff02f11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:41:35.619724 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf46898-afd3-47d6-b618-16685ff02f11 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:41:35.623296 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a39600-74c1-4c19-a3e0-f86b89f15e6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5d157abcf94745a5b92d6df64d872f73', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:41:35.623886 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a39600-74c1-4c19-a3e0-f86b89f15e6d service cinder] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:41:35.624163 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a39600-74c1-4c19-a3e0-f86b89f15e6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:41:35.624434 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a39600-74c1-4c19-a3e0-f86b89f15e6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:41:35.632874 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a39600-74c1-4c19-a3e0-f86b89f15e6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:41:35.634386 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a39600-74c1-4c19-a3e0-f86b89f15e6d service cinder] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:41:35.643179 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1480/2946] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:41:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:41:45.639111 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=26 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:41:46.675722 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729f191-8172-42fe-bbab-294280c68dde None None] REQUEST_METHOD: `POST` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:41:46.676284 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729f191-8172-42fe-bbab-294280c68dde None None] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:41:46.676394 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9729f191-8172-42fe-bbab-294280c68dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:41:46.704325 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.auth.core [None req-9729f191-8172-42fe-bbab-294280c68dde None None] MFA Rules not processed for user `8983f4ff039e4f6bbf01159accb1fb7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81627) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 31 19:41:46.705549 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-9729f191-8172-42fe-bbab-294280c68dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:41:46.725918 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1467/2947] 10.222.0.65 () {58 vars in 958 bytes} [Wed Aug 31 19:41:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 31 19:41:47.048367 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a39600-74c1-4c19-a3e0-f86b89f15e6d service cinder] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:41:47.058249 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a39600-74c1-4c19-a3e0-f86b89f15e6d service cinder] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:41:47.061779 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933af35-716e-4e42-b876-912630065d58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:41:47.062367 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933af35-716e-4e42-b876-912630065d58 service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:41:47.062645 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933af35-716e-4e42-b876-912630065d58 service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:41:47.062919 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6933af35-716e-4e42-b876-912630065d58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:41:47.070145 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933af35-716e-4e42-b876-912630065d58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:41:47.072095 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6933af35-716e-4e42-b876-912630065d58 service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:41:47.080073 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1481/2948] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:41:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:41:47.520785 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729f191-8172-42fe-bbab-294280c68dde None None] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:41:47.551583 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9729f191-8172-42fe-bbab-294280c68dde None None] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:41:47.557312 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc579e-fc4d-4857-9f70-8add6ee27ec4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:41:47.557928 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc579e-fc4d-4857-9f70-8add6ee27ec4 service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:41:47.558788 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc579e-fc4d-4857-9f70-8add6ee27ec4 service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:41:47.559079 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dc579e-fc4d-4857-9f70-8add6ee27ec4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:41:47.568130 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc579e-fc4d-4857-9f70-8add6ee27ec4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:41:47.569742 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dc579e-fc4d-4857-9f70-8add6ee27ec4 service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:41:47.580066 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1468/2949] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:41:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:41:48.045481 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933af35-716e-4e42-b876-912630065d58 service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:41:48.057604 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6933af35-716e-4e42-b876-912630065d58 service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:41:48.062534 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50cb07-f9ec-448c-af84-56475873afa3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:41:48.063122 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50cb07-f9ec-448c-af84-56475873afa3 service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:41:48.063433 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50cb07-f9ec-448c-af84-56475873afa3 service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:41:48.063704 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb50cb07-f9ec-448c-af84-56475873afa3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:41:48.072667 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50cb07-f9ec-448c-af84-56475873afa3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:41:48.075255 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb50cb07-f9ec-448c-af84-56475873afa3 service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:41:48.085967 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1482/2950] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:41:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:41:57.575601 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=26 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:41:58.081083 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=12 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:42:04.091309 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc579e-fc4d-4857-9f70-8add6ee27ec4 service placement] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:42:04.100834 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dc579e-fc4d-4857-9f70-8add6ee27ec4 service placement] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:42:04.104442 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151c6e0-ac8c-412e-b8ea-4429fd37758a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:42:04.105023 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151c6e0-ac8c-412e-b8ea-4429fd37758a service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:42:04.105335 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151c6e0-ac8c-412e-b8ea-4429fd37758a service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:42:04.105604 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151c6e0-ac8c-412e-b8ea-4429fd37758a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:42:04.114356 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151c6e0-ac8c-412e-b8ea-4429fd37758a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:42:04.115746 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5151c6e0-ac8c-412e-b8ea-4429fd37758a service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:42:04.125637 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1469/2951] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:42:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:42:14.121624 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=6 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:43:43.816631 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50cb07-f9ec-448c-af84-56475873afa3 service placement] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:43:43.857017 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb50cb07-f9ec-448c-af84-56475873afa3 service placement] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:43:43.860838 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5078c-0f68-415b-87f1-7ec350b9243c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:43:43.861470 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5078c-0f68-415b-87f1-7ec350b9243c service neutron] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:43:43.861749 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5078c-0f68-415b-87f1-7ec350b9243c service neutron] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:43:43.862038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd5078c-0f68-415b-87f1-7ec350b9243c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:43:43.869510 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd5078c-0f68-415b-87f1-7ec350b9243c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:43:43.870910 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd5078c-0f68-415b-87f1-7ec350b9243c service neutron] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:43:43.879275 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1483/2952] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:43:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:43:53.875722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=18 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:44:10.407839 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151c6e0-ac8c-412e-b8ea-4429fd37758a service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:44:10.437902 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151c6e0-ac8c-412e-b8ea-4429fd37758a service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:44:10.441437 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473062a7-4c01-4a8d-b204-9641bc1beb55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '093edc8c192c469f9532a45bb3326612', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:44:10.442029 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473062a7-4c01-4a8d-b204-9641bc1beb55 service neutron] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:44:10.442304 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473062a7-4c01-4a8d-b204-9641bc1beb55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:44:10.442570 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473062a7-4c01-4a8d-b204-9641bc1beb55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:44:10.501508 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473062a7-4c01-4a8d-b204-9641bc1beb55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:44:10.502961 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473062a7-4c01-4a8d-b204-9641bc1beb55 service neutron] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:44:10.512382 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1470/2953] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:44:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:44:13.602308 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5078c-0f68-415b-87f1-7ec350b9243c service neutron] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:44:13.626144 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd5078c-0f68-415b-87f1-7ec350b9243c service neutron] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:44:13.634722 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecc8a3-2204-4713-b0f8-db5b1c418869 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:44:13.635902 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecc8a3-2204-4713-b0f8-db5b1c418869 service placement] REQUEST_METHOD: `GET` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:44:13.636517 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecc8a3-2204-4713-b0f8-db5b1c418869 service placement] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:44:13.637126 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecc8a3-2204-4713-b0f8-db5b1c418869 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:44:13.640137 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-f0ecc8a3-2204-4713-b0f8-db5b1c418869 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:44:13.677473 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecc8a3-2204-4713-b0f8-db5b1c418869 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:44:13.679116 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecc8a3-2204-4713-b0f8-db5b1c418869 service placement] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:44:13.697413 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1484/2954] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:44:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:44:16.597125 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473062a7-4c01-4a8d-b204-9641bc1beb55 service neutron] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:44:16.607056 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473062a7-4c01-4a8d-b204-9641bc1beb55 service neutron] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:44:16.612181 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a687953-31d3-4130-b8f9-c49c63fda1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:44:16.613392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a687953-31d3-4130-b8f9-c49c63fda1da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:44:16.613392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a687953-31d3-4130-b8f9-c49c63fda1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:44:16.613392 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a687953-31d3-4130-b8f9-c49c63fda1da admin admin] PATH_INFO: `/v3/users/8983f4ff039e4f6bbf01159accb1fb7c` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:44:16.614534 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a687953-31d3-4130-b8f9-c49c63fda1da admin admin] RBAC: Authorizing `identity:delete_user(user_id=8983f4ff039e4f6bbf01159accb1fb7c)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:44:16.616443 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a687953-31d3-4130-b8f9-c49c63fda1da admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:44:16.664825 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2a687953-31d3-4130-b8f9-c49c63fda1da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8983f4ff039e4f6bbf01159accb1fb7c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:16.671755 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2a687953-31d3-4130-b8f9-c49c63fda1da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8983f4ff039e4f6bbf01159accb1fb7c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:16.683926 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2a687953-31d3-4130-b8f9-c49c63fda1da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8983f4ff039e4f6bbf01159accb1fb7c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:16.684667 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2a687953-31d3-4130-b8f9-c49c63fda1da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8983f4ff039e4f6bbf01159accb1fb7c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:16.685296 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-2a687953-31d3-4130-b8f9-c49c63fda1da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8983f4ff039e4f6bbf01159accb1fb7c'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:16.698687 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1471/2955] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:44:16 2022] DELETE /identity/v3/users/8983f4ff039e4f6bbf01159accb1fb7c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:44:16.870797 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecc8a3-2204-4713-b0f8-db5b1c418869 service placement] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:44:16.881974 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecc8a3-2204-4713-b0f8-db5b1c418869 service placement] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:44:16.937108 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f076ac-debc-4402-abb4-bd3c014e3a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:44:16.937759 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f076ac-debc-4402-abb4-bd3c014e3a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:44:16.938038 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f076ac-debc-4402-abb4-bd3c014e3a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:44:16.938338 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f076ac-debc-4402-abb4-bd3c014e3a08 admin admin] PATH_INFO: `/v3/projects/2f7614eeafc645958e7308586df90b8e` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:44:16.940022 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f076ac-debc-4402-abb4-bd3c014e3a08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f7614eeafc645958e7308586df90b8e)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:44:16.941953 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f076ac-debc-4402-abb4-bd3c014e3a08 admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:44:16.988182 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e4f076ac-debc-4402-abb4-bd3c014e3a08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f7614eeafc645958e7308586df90b8e', 'request_id': 'req-e4f076ac-debc-4402-abb4-bd3c014e3a08'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:16.994155 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e4f076ac-debc-4402-abb4-bd3c014e3a08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f7614eeafc645958e7308586df90b8e', 'request_id': 'req-e4f076ac-debc-4402-abb4-bd3c014e3a08'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:16.999679 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e4f076ac-debc-4402-abb4-bd3c014e3a08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f7614eeafc645958e7308586df90b8e', 'request_id': 'req-e4f076ac-debc-4402-abb4-bd3c014e3a08'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:17.011708 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e4f076ac-debc-4402-abb4-bd3c014e3a08 admin admin] The token cache is being invalidate because project 2f7614eeafc645958e7308586df90b8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:44:17.011994 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e4f076ac-debc-4402-abb4-bd3c014e3a08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:17.012603 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-e4f076ac-debc-4402-abb4-bd3c014e3a08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:17.014135 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1485/2956] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:44:16 2022] DELETE /identity/v3/projects/2f7614eeafc645958e7308586df90b8e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:44:17.018220 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a687953-31d3-4130-b8f9-c49c63fda1da admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:44:17.019618 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-2a687953-31d3-4130-b8f9-c49c63fda1da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:44:17.032419 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a687953-31d3-4130-b8f9-c49c63fda1da admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:44:17.075579 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:44:17.089968 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937a32c-d1c3-473c-bc08-f0962f736ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:44:17.090590 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937a32c-d1c3-473c-bc08-f0962f736ba8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:44:17.090869 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937a32c-d1c3-473c-bc08-f0962f736ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:44:17.091146 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1937a32c-d1c3-473c-bc08-f0962f736ba8 admin admin] PATH_INFO: `/v3/users/94784a7747294ced93016b7a7595ad86` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:44:17.092262 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937a32c-d1c3-473c-bc08-f0962f736ba8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94784a7747294ced93016b7a7595ad86)` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:44:17.094216 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1937a32c-d1c3-473c-bc08-f0962f736ba8 admin admin] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:44:17.139864 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-1937a32c-d1c3-473c-bc08-f0962f736ba8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94784a7747294ced93016b7a7595ad86'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:17.147358 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-1937a32c-d1c3-473c-bc08-f0962f736ba8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94784a7747294ced93016b7a7595ad86'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:17.158828 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-1937a32c-d1c3-473c-bc08-f0962f736ba8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94784a7747294ced93016b7a7595ad86'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:17.159498 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-1937a32c-d1c3-473c-bc08-f0962f736ba8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94784a7747294ced93016b7a7595ad86'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:17.160111 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.notifications [None req-1937a32c-d1c3-473c-bc08-f0962f736ba8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94784a7747294ced93016b7a7595ad86'} {{(pid=81627) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:17.172802 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1472/2957] 10.222.0.65 () {58 vars in 1224 bytes} [Wed Aug 31 19:44:17 2022] DELETE /identity/v3/users/94784a7747294ced93016b7a7595ad86 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 31 19:44:17.345927 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f076ac-debc-4402-abb4-bd3c014e3a08 admin admin] Authenticating user token {{(pid=81626) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:44:17.347341 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.fernet_utils [None req-e4f076ac-debc-4402-abb4-bd3c014e3a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81626) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:44:17.359726 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f076ac-debc-4402-abb4-bd3c014e3a08 admin admin] Validating token access rules against request {{(pid=81626) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:44:17.416213 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5362b36-32ea-4c2d-ae2a-7d58fd52e8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2d817dbb81e84447ac0c66c6621a6e8f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8d8519390e3f4536964a76bc41b6e43c', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81626) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:44:17.416877 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5362b36-32ea-4c2d-ae2a-7d58fd52e8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:44:17.417152 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5362b36-32ea-4c2d-ae2a-7d58fd52e8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:44:17.417447 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5362b36-32ea-4c2d-ae2a-7d58fd52e8be admin admin] PATH_INFO: `/v3/projects/33383c2ac29b4582a33f096cfb798d57` {{(pid=81626) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:44:17.418575 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5362b36-32ea-4c2d-ae2a-7d58fd52e8be admin admin] RBAC: Authorizing `identity:delete_project(project_id=33383c2ac29b4582a33f096cfb798d57)` {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:44:17.420495 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5362b36-32ea-4c2d-ae2a-7d58fd52e8be admin admin] RBAC: Authorization granted {{(pid=81626) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:44:17.428882 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:44:17.468657 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a5362b36-32ea-4c2d-ae2a-7d58fd52e8be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33383c2ac29b4582a33f096cfb798d57', 'request_id': 'req-a5362b36-32ea-4c2d-ae2a-7d58fd52e8be'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:17.474648 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a5362b36-32ea-4c2d-ae2a-7d58fd52e8be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33383c2ac29b4582a33f096cfb798d57', 'request_id': 'req-a5362b36-32ea-4c2d-ae2a-7d58fd52e8be'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:17.480696 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a5362b36-32ea-4c2d-ae2a-7d58fd52e8be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33383c2ac29b4582a33f096cfb798d57', 'request_id': 'req-a5362b36-32ea-4c2d-ae2a-7d58fd52e8be'} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:17.494875 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a5362b36-32ea-4c2d-ae2a-7d58fd52e8be admin admin] The token cache is being invalidate because project 33383c2ac29b4582a33f096cfb798d57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81626) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 31 19:44:17.495167 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a5362b36-32ea-4c2d-ae2a-7d58fd52e8be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:17.495792 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG keystone.notifications [None req-a5362b36-32ea-4c2d-ae2a-7d58fd52e8be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81626) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 31 19:44:17.497381 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: [pid: 81626|app: 0|req: 1486/2958] 10.222.0.65 () {58 vars in 1233 bytes} [Wed Aug 31 19:44:17 2022] DELETE /identity/v3/projects/33383c2ac29b4582a33f096cfb798d57 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 31 19:44:27.168906 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=32,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:44:27.493365 dv-cinder-835384-26-iscsi devstack@keystone.service[81626]: DEBUG dbcounter [-] [81626] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81626) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 31 19:47:02.112491 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937a32c-d1c3-473c-bc08-f0962f736ba8 admin admin] Authenticating user token {{(pid=81627) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 31 19:47:02.113931 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-1937a32c-d1c3-473c-bc08-f0962f736ba8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:47:02.182251 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1937a32c-d1c3-473c-bc08-f0962f736ba8 admin admin] Validating token access rules against request {{(pid=81627) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 31 19:47:02.241826 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c755c9-bb29-4c8f-9ea9-24f06e9299f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '74f1846e395e4d5483fcbb41eb28a77d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fdc1fee7aedd4f4a9fba0fd6fb614e33', 'project_domain_id': 'default', 'roles': ['service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81627) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 31 19:47:02.242454 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c755c9-bb29-4c8f-9ea9-24f06e9299f9 service placement] REQUEST_METHOD: `GET` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 31 19:47:02.242734 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c755c9-bb29-4c8f-9ea9-24f06e9299f9 service placement] SCRIPT_NAME: `/identity` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 31 19:47:02.243004 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c755c9-bb29-4c8f-9ea9-24f06e9299f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81627) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 31 19:47:02.244741 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.fernet_utils [None req-92c755c9-bb29-4c8f-9ea9-24f06e9299f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81627) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 31 19:47:02.255895 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c755c9-bb29-4c8f-9ea9-24f06e9299f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 31 19:47:02.257739 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c755c9-bb29-4c8f-9ea9-24f06e9299f9 service placement] RBAC: Authorization granted {{(pid=81627) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 31 19:47:02.267879 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: [pid: 81627|app: 0|req: 1473/2959] 10.222.0.65 () {60 vars in 1365 bytes} [Wed Aug 31 19:47:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 31 19:47:12.263106 dv-cinder-835384-26-iscsi devstack@keystone.service[81627]: DEBUG dbcounter [-] [81627] Writing DB stats keystone:SELECT=58 {{(pid=81627) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}